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Abstract 


The  U.S.  defense  acquisition  workforce  (DAW)  experienced  significant  personnel  reductions 
in  the  1990s,  leading  to  concerns  regarding  the  size  of  and  age  distribution  within  the  workforce 
in  the  early  2000s.  Defense  officials  responded  to  these  concerns,  instituting  a  DAW  “growth 
initiative.”  The  growth  initiative  was  successful  in  increasing  the  size  of  the  DAW,  but  little  is 
known  about  how  the  hiring  surge  changed  who  the  DAW  hired  into  its  ranks,  and  how  this 
might  influence  workforce  outcomes,  such  as  retention.  This  dissertation  examines  this  issue  by 
focusing  on  the  civilian  portion  of  the  DAW  (civilian  DAW)  and  on  entrants’  prior  work 
experience. 

This  dissertation  utilizes  DoD  personnel  data  to  (1)  describe  civilian  DAW  cohorts  in  terms 
of  past  work  experience  and  illustrate  how  the  hiring  surge  has  changed  cohort  past-work- 
experience  characteristics;  (2)  evaluate  how  prior  work  experience  relates  to  retention  in  the 
civilian  DAW;  and  (3)  estimate  how  the  growth  initiative  has  influenced  overall  cohort-level 
retention  rates. 

The  analyses  reveal  that  the  growth  initiative  was  fueled  mainly  by  outside  hires  with  no 
prior  DoD  experience,  and  some  evidence  suggests  that  these  DoD  newcomers — in  general — 
tend  to  have  the  highest  retention  in  the  civilian  DAW.  Additionally,  the  analyses  reveal  that 
internal  hires  are  more  attached  to  the  DoD  civilian  workforce  than  are  external  hires.  In  line 
with  these  conclusions,  the  synthetic-cohort  analysis  finds  that  the  hiring  surge  likely  produced 
cohorts  with  greater  civilian  DAW  retention. 
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Chapter  1 .  Introduction 


The  U.S.  Department  of  Defense  (DoD)  relies  on  its  acquisition  process,  and  the  workforce 
that  supports  that  process,  to  provide  the  weapon  systems  and  services  it  needs  to  execute  its 
ultimate  job — ensuring  the  security  of  the  American  people  (Gates  et  al.,  2013).  This  is  a 
significant  task,  as  defense  acquisition  professionals  are  responsible  for  allocating  hundreds  of 
billions  of  federal  dollars  annually  (Office  of  the  Under  Secretary  of  Defense,  2015)  and  for 
equipping  U.S.  military  members  in  the  field  with  the  tools  they  require  to  survive  and  complete 
their  missions.  Given  the  significant  financial  and  security  responsibilities,  the  defense 
acquisition  workforce  (DAW)  has  been  closely  scrutinized  for  years  and  has  undergone  various 
“rightsizing”1  and  professionalization  initiatives. 

Most  recently,  the  DAW  underwent  an  ambitious  growth  initiative.  In  the  early  2000s,  there 
were  concerns  about  the  state  of  the  DAW.  First,  the  workforce  was  aging  (U.S.  Department  of 
Defense,  2000;  Testimony  by  Dr.  Diane  M.  Disney,  2000),  and  reports  warned  that  the  DAW 
could  experience  significant  attrition  in  the  subsequent  years  (Office  of  the  Inspector  General, 
2000;  U.S.  Department  of  Defense,  2000;  Office  of  the  Secretary  of  Defense,  2000). 
Additionally,  a  report  from  the  Office  of  the  Inspector  General  stated  that  the  personnel 
reductions  that  took  place  throughout  the  1990s  made  it  difficult  for  acquisition  professionals  to 
meet  the  demands  for  their  services  (Office  of  the  Inspector  General,  2000).  In  response  to  these 
concerns,  legislators  established  the  Defense  Acquisition  Workforce  Development  Fund 
(DAWDF)  in  the  FY  2008  National  Defense  Authorization  Act  (NDAA)  (Public  Law  110-181, 
National  Defense  Authorization  Act  for  Fiscal  Year  2008)  and  granted  Expedited  Hiring 
Authority  in  the  FY  2009  NDAA  for  a  number  of  acquisition  positions  (Public  Law  1 10-417, 
Duncan  Hunter  National  Defense  Authorization  Act  for  Fiscal  Year  2009).  The  DAWDF  created 
a  pool  of  protected  resources  that  officials  could  use  only  for  growing  and  developing  the  DAW 
(Public  Law  1 10-181,  National  Defense  Authorization  Act  for  Fiscal  Year  2008;  Title  10  U.S.C. 
Ch.  87  Section  1705),  and  the  Expedited  Hiring  Authority  helped  streamline  the  actual  hiring 
process  (Public  Law  1 10-417,  Duncan  Hunter  National  Defense  Authorization  Act  for  Fiscal 
Year  2009;  Title  10  U.S.C.  Ch.  87  Section  1705(g);  Title  5  U.S.C.  Ch.  33  Section  3304;  Title  5 
U.S.C.  Ch.  53  Section  5333;  Title  5  U.S.C.  Ch.  57  Section  5753). 2  In  April  of  2009,  Defense 


1  Gates  et  al.  (2008)  write,  “A  popular  term  used  in  discussing  the  management  of  government  agencies  is 
‘rightsizing’ — having  the  optimal  or  appropriate  number  of  employees  to  accomplish  the  required  tasks”  (p.  3).  This 
study  also  uses  the  term  “rightsizing.” 

“  The  FY  2009  NDAA  ends  the  Expedited  Hiring  Authority  for  the  DAW  at  the  end  of  FY  2012  (Public  Law  110- 
417,  Duncan  Hunter  National  Defense  Authorization  Act  for  Fiscal  Year  2009).  However,  the  Authorization  has 
since  been  extended  to  the  end  of  FY  2017  (Public  Law  112-239,  National  Defense  Authorization  Act  for  Fiscal 
Year  2013). 
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Secretary  Robert  Gates  announced  a  plan  to  increase  the  size  of  the  DAW  by  20,000  by  FY  2015 
(Gates,  2009a). 

Personnel  managers  and  the  DAW  as  a  whole  responded  quickly,  with  help  from  the 
DAWDF  and  the  Expedited  Hiring  Authority.  In  fact,  the  DAW  neared  its  growth  initiative  goals 
more  quickly  than  expected.  From  the  end  of  FY  2009  to  the  end  of  FY  2011,  the  DAW  grew 
from  approximately  133,000  personnel  to  just  under  152,000 — an  increase  of  over  18,000,  or 
around  fourteen  percent  (see  Figure  1.1)  (Office  of  the  Under  Secretary  of  Defense  for 
Acquisition,  2015). 


Figure  1.1.  Total  Size  of  the  DAW  Increased  Quickly  Between  End-of-FY  2008  and  End-of-FY  2011 
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SOURCE:  (Office  of  Human  Capital  Initiatives) 

Figure  1 . 1  shows  that  the  growth  initiative  was  effective  in  quickly  growing  the  overall  size 
of  the  DAW,  but  it  also  raises  the  following  questions  regarding  what  type  of  individuals  were 
brought  into  the  workforce  during  the  initiative  and  what  implications  these  changes  have  on 
longer-term  retention. 


1 .  What  are  the  characteristics  of  the  individuals  who  were  brought  into  the  DAW  during 
the  hiring  surge? 

2.  How  are  they  different  from  those  who  were  hired  in  the  years  prior  to  the  growth 
initiative? 
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3.  How  might  any  differences  in  these  cohort  groups  affect  workforce  outcomes,  such  as 
retention? 

This  study  addresses  these  questions  by  focusing  on  the  civilian  DAW  and  examining 
entrants’  prior  work  experience.  “Organizational  socialization”  theory  suggests  that  prior 
experiences  influence  how  an  individual  assimilates  into  a  new  work  environment  (Bauer  et  ah, 
2007,  p.  7;  Saks  and  Ashforth,  1997,  p.  234;  Adkins,  1995,  p.  839;  Carr  et  ah,  2006,  p.  343;  Van 
Maanen  and  Shein,  1977,  p.  1).  Also,  this  is  a  natural  place  to  start  because  prior  work 
experience  heavily  influences  the  employee  selection  process  (Dokko,  Wilk  and  Rothbard,  2009; 
Adkins,  1995).  Past  work  experience  is  a  large  part  of  each  applicant’s  resume  or  job 
application — it  signals  what  “skills”  and  “habits”  each  applicant  has  (Dokko,  Wilk  and  Rothbard, 
2009,  p.  52). 

The  official  DAW  is  made  up  of  DoD  civil  servants,  active-duty  military  members,  and 
contractors.  Data  on  the  contractor  workforce,  however,  are  limited  (Gates,  2009b,  pp.  14-15),  so 
empirical  examinations  of  the  DAW  tend  to  be  confined  to  the  acquisition  personnel  who  are 
employed  and  tracked  by  the  DoD — the  DoD  civilian  employees  and  the  active-duty  military 
members.  These  personnel  make  up  what  is  called  the  “organic”  DAW  (Gates  et  ah,  2013,  pp.  xi, 
1).  In  this  document,  the  term  DAW  refers  strictly  to  these  organic  defense  acquisition 
employees  (see  Figure  1.2). 

Figure  1.2.  The  Organic  DAW  Consists  of  Active-Duty  Military  and  DoD  Civilian  Acquisition 

Professionals 
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Military  DAW  - 15,351  (fy  15) 

Civilian  DAW  - 140,962  (fyis) 


SOURCES:  DAW  numbers  come  from  the  Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology  and 
Logistics  -  Human  Capital  Initiatives  (Office  of  Human  Capital  Initiatives).  The  active-duty  military  count  comes  from 
the  end-of-FY  Active  Duty  Master  File,  and  the  DoD  civilian  count  comes  from  the  end-of-FY  DoD  Civilian  Personnel 
Inventory  File.  RAND  receives  these  files  from  the  Defense  Manpower  Data  Center.  More  detail  on  the  personnel 
included  in  these  files  can  be  found  in  Chapter  6.  Data  and  Definitions. 
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Figure  1.3.  The  Vast  Majority  of  the  Organic  DAW  Consists  of  Civilian  Personnel' 


"fli-oamr"  HAW  hu  Pprcnnnpl  Tvnp  FV  701 


Army  Navy  Air  Force  Other 


Q  Military 
■  Civilian 


NOTE:  Navy  includes  Marine  Corps 
SOURCE:  Office  of  Human  Capital  Initiatives  (2015) 

This  analysis  focuses  on  the  civilian  DAW.  Civil  servants  make  up  approximately  90  percent 
of  the  organic  DAW  (Office  of  Human  Capital  Initiatives,  2015)  (see  Figure  1.3  and  Table  1.1), 
and  the  hiring  and  management  of  this  segment  of  the  acquisition  workforce  is  of  critical 
importance.  Because  civilian  defense  acquisition  employees  can  have  a  diverse  set  of  prior 
workforce  experiences  upon  entry  into  the  DAW,  analysis  can  contribute  to  a  better 
understanding  of  the  relationship  between  those  prior  experiences  and  workforce  outcomes.  In 
contrast,  military  members  typically  join  the  military  as  a  junior  officer  or  enlisted  member  and 
work  their  way  up  through  the  ranks.  Therefore,  many  individuals  in  the  military  DAW  likely 
have  experience  in  only  one  DoD  workforce — the  wider  active  duty  military. 

Table  1.1.  This  Study  Focuses  on  Analyzing  Entrants  into  the  Civilian  DAW 


Total  DAW 

"Organic"  DAW 

Analysis  Population 

Civilian  DAW 

Civilian  DAW 

Civilian  DAW 

Military  DAW 

Military  DAW 

- 

Gov't  Contractors 

- 

- 

3 

Other  RAND  documents  have  used  a  similar  chart  to  describe  the  DAW.  See,  for  example,  Gates  et  al.  (2008)  and 
Gates  et  al.  (2013). 


4 


Looking  at  the  civilian  DAW  cohorts  from  FY  2000  to  FY  2014,  one  notes  a  large  increase  in 
the  number  of  entrants  during  FYs  2008-2011  (see  Figure  1.4,  below).  These  cohorts  are  referred 
to  as  the  “growth  initiative  cohorts”  or  the  “hiring  surge  cohorts.” 


Figure  1.4.  Civilian  DAW  Cohort  Size  Increased  Significantly  in  FYs  2008-2011 
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In  order  to  (1)  examine  how  the  growth  initiative  changed  workforce  characteristics  and  (2) 
evaluate  how  these  changes  might  influence  overall  retention  trends,  this  study  examines  the 
civilian  DAW  and  the  growth  initiative  policies  through  the  lens  of  prior  work  experience  and 
addresses  the  following  research  questions: 

1 .  How  did  the  growth  initiative  change  whom  the  civilian  DAW  hired  into  its  ranks,  in 
terms  of  prior  work  experience? 

2.  How  does  entrants’  past  work  experience  relate  to  civilian  DAW  retention? 

3.  What  does  this  suggest  in  terms  of  how  the  growth  initiative  may  have  influenced  overall 
cohort-level  retention  in  the  civilian  DAW? 

Past  analyses  provide  acquisition  officials  insight  into  the  makeup  of  the  DAW,* 4  and  a  recent 
study  examines — more  broadly — the  relationship  between  various  personnel  factors  and 
workforce  outcomes.5  But  relatively  little  is  known  about  the  past-work-experience 


These  past  analyses  (Gates  et  al.,  2008;  Gates  et  ah,  2009;  Gates  et  ah,  2013)  are  reviewed  in  “Chapter  3.  Prior 

DAW-Related  Studies.” 

5  This  study  (Guo,  Hall-Partyka  and  Gates,  2014)  is  also  reviewed  in  “Chapter  3.  Prior  DAW-Related  Studies.” 
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characteristics  of  DAW  entrants  and  how  different  past  work  experiences  relate  to  DAW 
outcomes,  such  as  retention.  Understanding  how  the  hiring  surge  changed  cohorts’  prior- work- 
experience  characteristics  and  how  this  shift  may  influence  workforce  outcomes  would  provide 
defense  officials  with  the  knowledge  to  appreciate  how  “rightsizing”  initiatives,  such  as  the  most 
recent  growth  initiative,  have  secondary  implications.  With  this  knowledge,  decision  makers  can 
build  more  effective  workforce  management  policies  and  plan  for  outcome  shifts  as  a  result  of 
these  policies. 

This  study  utilizes  DoD  personnel  data  to  (1)  describe  civilian  DAW  cohorts,  in  terms  of  past 
work  experience,  and  illustrate  how  the  hiring  surge  has  changed  cohort  past-work-experience 
characteristics;  (2)  evaluate  how  prior  work  experience  relates  to  retention  in  the  civilian  DAW; 
and  (3)  estimate  how  the  growth  initiative  has  influenced  overall  cohort-level  retention  rates. 
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Chapter  2.  Background:  The  DAW  and  the  Growth  Initiative 


The  DAW  is  a  large  and  broad  group  of  professionals.  This  chapter  describes  the  workforce, 
reviews  how  it  was  established  and  how  it  is  tracked,  and  outlines  the  events  leading  up  to  and 
the  policies  involved  in  the  growth  initiative. 

Establishing,  Professionalizing,  and  Tracking  the  DAW 

The  Defense  Acquisition  Workforce  Improvement  Act  (DAWIA)  created  and  shaped  the 
official  DAW  that  exists  today  (Layton,  2007;  Gates,  2009b).  The  legislation  orders  the  DoD  to 
track  its  acquisition  employees  and  create  and  enforce  professionalization  standards.  However, 
given  the  size  of  the  DoD  and  the  variety  of  roles  acquisition  professionals  play  in  the  acquisition 
process,  counting  and  tracking  the  DAW  has  not  been  straightforward.  In  fact,  there  have  been 
multiple  different  methods  used  for  tracking  these  acquisition  employees.  The  DAWIA  and  these 
various  counting  methods  are  explained  in  more  detail,  below. 

The  Packard  Commission  and  the  Defense  Acquisition  Workforce  Improvement  Act 
(DAWIA) 

DAWIA  was  largely  a  response  to  the  controversies  that  surrounded  defense  acquisition 
throughout  the  1980s  (Gates,  2009b).  Allegations  of  “fraud,  waste,  and  mismanagement”  (Fox  et 
ah,  201 1,  p.  120)  surfaced  in  the  early-  to  mid-1980s,  prompting  an  official  review:  The  Packard 
Commission  review  (Layton,  2007;  Gates,  2009b).  The  final  Packard  Commission  report  makes 
recommendations  for  many  parts  of  the  DoD,  including  defense  acquisition  (U.S.  President's 
Blue  Ribbon  Commission  on  Defense  Management,  1986).  In  regard  to  the  DAW,  the  report 
recommended  that  the  DoD  “establish  flexible  personnel  management  policies”  and  “establish 
business-related  education  and  experience  criteria  for  civilian  contracting  personnel,  which  will 
provide  a  basis  for  the  professionalization  of  their  career  paths”  (U.S.  President’s  Blue  Ribbon 
Commission  on  Defense  Management,  1986,  p.  880  (pdf)). 

DAWIA  became  law  in  1990  (Title  10  U.S.C.  Ch.  87),  and  the  legislation  includes  mandates 
that  resemble  the  recommendations  made  by  the  Packard  Commission  report.  DAWIA  mandated 
standardized  acquisition  workforce  policy  across  the  DoD  (Title  10  U.S.C.  Ch.  87  Section  1701). 
In  particular,  among  other  things,  DAWIA: 

•  Mandated  the  “Designation  of  acquisition  positions”  (Title  10  U.S.C.  Ch.  87  Section 
1721)  and  the  creation  of  an  “Acquisition  Corps”  (Title  10  U.S.C.  Ch.  87  Section  1731) 

•  Directed  defense  officials  to  “establish  education,  training,  and  experience  requirements 
for  each  acquisition  position,  based  on  the  level  of  complexity  of  duties  carried  out  in  the 
position”  (Title  10  U.S.C.  Ch.  87  Section  1723) 


7 


•  Required  the  DoD  to  “ensure  that  appropriate  career  paths”  exist  for  acquisition 
professionals  (Title  10  U.S.C.  Ch.  87  Section  1722) 

•  Mandated  the  establishment  of  a  “Defense  Acquisition  University”  (Title  10  U.S.C.  Ch. 
87  Section  1746) 

•  “Required]  th[e]  DoD  [to]  count  and  track  the  size  of  the  [D]AW”  (Gates,  2009b,  pp.  7- 
8;  Title  10  U.S.C.  Ch.  87) 

Counting  and  Tracking  the  DAW 

Tracking  the  size  of  the  acquisition  workforce  is  no  straightforward  task  as  acquisition 
professionals  are  spread  throughout  the  DoD  and  play  a  variety  of  different  roles.  Officials  have 
been  attempting  to  gauge  the  size  of  the  DAW  for  decades  (Defense  Acquisition  University, 
2007;  Grasso,  1999).  During  that  time,  several  counting  methodologies  have  been  used.  This 
study  uses  what  is  called  the  “DAWIA  Count.”  This  counting  methodology,  along  with  two 
others  that  have  been  used,  is  described  below.6 

First,  the  Packard  Commission,  in  its  review  of  the  DoD  in  the  1980s,  developed  the 
“Packard  Count”  or  the  “Acquisition  Organization  Count”  (Defense  Acquisition  University, 
2007,  p.  3-5;  United  States  Department  of  Defense  Office  of  Inspector  General,  2006;  Gates  et 
ah,  2008;  Gates  et  ah,  2013).  The  acquisition  organization  count  simply  includes  all  individuals 
who  work  in  one  of  the  official  acquisition  organizations  (Defense  Acquisition  University,  2007; 
United  States  Department  of  Defense  Office  of  Inspector  General,  2006;  Gates  et  ah,  2008; 

Gates  et  ah,  20 13). 7  This  counting  method  is  simple  but  inaccurate  if  officials  are  attempting  to 
track  the  number  of  personnel  who  are  “directly  involved”  in  the  acquisition  process:  It  includes 
support  personnel  who  work  in  acquisition  organizations  but  do  not  have  acquisition 
responsibilities  and  does  not  include  acquisition  professionals  who  fall  outside  of  the  official 
acquisition  organizations  (Defense  Acquisition  University,  2007,  p.  3-5). 

The  “DAWIA  Count”  (Defense  Acquisition  University,  2007,  p.  3-6),  which  started  in  FY 
1992  (Defense  Acquisition  University,  2007),  stems  from  DAWIA’s  mandate  to  track  and  count 
all  acquisition  personnel  and  create  official  defense  acquisition  career  fields  (U.S.  Department  of 
Defense,  2010).  The  count  is  position  based:  Officials  designate  certain  positions  to  be 
acquisition  positions — based  on  the  amount  of  acquisition  work  that  position  does  (Defense 
Acquisition  University,  2007;  U.S.  Department  of  Defense,  2010;  Gates  et  ah,  2008).  And  each 
acquisition  position  is  assigned  to  an  acquisition  career  field  based  on  the  nature  of  the 
acquisition  work  (Defense  Acquisition  University,  2007;  U.S.  Department  of  Defense,  2010). 
Under  the  DAWIA  Count,  the  individuals  who  work  in  these  acquisition-designated  positons  are 
counted  in  the  DAW  (Defense  Acquisition  University,  2007;  U.S.  Department  of  Defense,  2010; 

6  The  following  summary  of  the  various  DAW  counts  is  based  on  descriptions  from  previous  reports  (Defense 
Acquisition  University,  2007;  United  States  Department  of  Defense  Office  of  Inspector  General,  2006;  U.S. 
Department  of  Defense,  2010;  Grasso,  1999;  Gates  et  al.,  2013;  Gates  et  al.,  2008). 

7  The  original  acquisition  organizations  are  outlined  in  DoDI  5000.55  (U.S.  Department  of  Defense,  1991,  pp.  19- 
20;  United  States  Department  of  Defense  Office  of  Inspector  General,  2006,  pp.  25-26). 
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Gates  et  al.,  2008).  The  DAWIA  legislation  states  that  the  DAWIA  Count  should  “be 
implemented  uniformly”  across  the  DoD  (Defense  Acquisition  University,  2007,  p.  3-7), 8  but  the 
ultimate  implementation  was  and  is  left  up  to  the  individual  DoD  components  (U.S.  Department 
of  Defense,  20 10). 9 

A  third  counting  methodology,  referred  to  as  the  “Refined  Packard  [Cjounf  ’  was  developed 
in  the  late  1990s  (Defense  Acquisition  University,  2007,  pp.  3-9,  xii;  Burman,  Cavallini  and 
Harris,  1999;  United  States  Department  of  Defense  Office  of  Inspector  General,  2006).  Under 
this  methodology,  there  are  three  categories  included  in  the  DAW  count:  Category  I  consists  of 
individuals  in  certain  occupations,  “such  as  contracting”  that  are  acquisition  in  nature  (Defense 
Acquisition  University,  2007,  p.  3-6;  Burman,  Cavallini  and  Harris,  1999;  United  States 
Department  of  Defense  Office  of  Inspector  General,  2006,  p.  8;  Gates  et  al.,  2008).  Category  II 
includes  individuals  in  certain  occupations  only  if  the  individuals  work  in  an  acquisition 
organization  (Defense  Acquisition  University,  2007;  Burman,  Cavallini  and  Harris,  1999;  United 
States  Department  of  Defense  Office  of  Inspector  General,  2006;  Gates  et  al.,  2008).  And  the 
third  category  (Category  III)  is  the  adjustment  category.  Under  this  category,  components  can 
add  personnel  to  or  remove  them  from  the  DAW  count  if  they  believe  these  individuals  should  or 
should  not  be  counted  (Defense  Acquisition  University,  2007;  Burman,  Cavallini  and  Harris, 
1999;  United  States  Department  of  Defense  Office  of  Inspector  General,  2006;  Gates  et  al., 

2008). 

The  Refined  Packard  Count  began  in  FY  1998,  and  the  DAWIA  Count  continued  as  a 
parallel  counting  method  (Defense  Acquisition  University,  2007;  U.S.  Department  of  Defense, 
2010;  Gates  et  al.,  2013).  The  two  counts  “converged”  over  the  next  several  years  “through  a 
process  called... ‘assimilation’,”  which  involved  improving  the  “consistency”  with  which 
positions  were  included  in  the  DAW  (Defense  Acquisition  University,  2007,  p.  3-7;  U.S. 
Department  of  Defense,  2010;  Under  Secretary  of  Defense  for  Acquisition,  2001;  Gates  et  al., 
2013).  By  FY  2005,  only  the  DAWIA  Count  remained  (in  its  revised  form)  (Defense  Acquisition 
University,  2007;  U.S.  Department  of  Defense,  2010;  Gates  et  al.,  2008;  Gates  et  al.,  2013). 

There  is  no  perfect  way  to  count  and  track  the  DAW.  This  study  uses  the  DAWIA  Count  data 
going  back  to  FY  1992.  It  is  important  to  recognize  that  the  implementation  of  this  count  has 
changed  over  the  years,  as  described  above. 

An  Overview  of  the  Current  DAW 

The  DAW  represents  a  significant  portion  of  DoD  personnel  and  includes  a  diverse  range  of 
careers.  The  vast  majority  of  the  DAW  positions  are  filled  by  DoD  civilians,  although  military 
personnel  also  play  a  significant  role.  As  of  the  end  of  fiscal  year  (FY)  2015,  there  were  140,962 


8  See  sections  1701  and  1721  in  DAWIA  (Title  10  U.S.C.  Ch.  87  Section  1701;  Title  10  U.S.C.  Ch.  87  Section 
1721). 

9  See  DoDI  5000.58  (U.S.  Department  of  Defense,  1992)  and  DoDD  5000.52  (U.S.  Department  of  Defense,  2005). 
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civilian  DAW  personnel  and  15,351  military  DAW  personnel  (Office  of  Human  Capital 
Initiatives,  2015).  The  majority  of  these  acquisition  employees  work  under  one  of  the  three  major 
military  service  organizations:  the  Department  of  the  Anny,  the  Department  of  the  Navy  (which 
includes  the  Marine  Corps),  and  the  Department  of  the  Air  Force.  However,  there  is  also  a 
significant  portion  of  DAW  civilians  that  work  for  other  organizations  within  the  DoD,  i.e.  the 
defense  agencies  (Office  of  Human  Capital  Initiatives,  2015;  Gates  et  al.,  2013).  Figure  2.1, 
below,  illustrates  the  distribution  of  the  acquisition  workforce  among  the  defense  organizations. 

Figure  2.1.  The  DAW  Distribution  by  DoD  Organization,  FY  201 510 
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NOTE:  The  Department  of  the  Navy  includes  the  Marine  Corps 
SOURCE:  Office  of  Human  Capital  Initiatives  (2015) 

Because  of  the  range  of  responsibilities  for  the  DAW,  the  acquisition  community  is 
composed  of  a  diverse  set  of  career  fields.  They  have  varied  throughout  the  years  (Gates  et  al., 
2013;  Guo,  Hall-Partyka  and  Gates,  2014),  but  at  the  end  of  FY  2015  there  were  14  career  fields 
in  the  DAW  (Office  of  Human  Capital  Initiatives,  2015;  Gates  et  al.,  2013;  Guo,  Hall-Partyka 
and  Gates,  20 14):*  11 

•  Auditing 

•  Business  -  Cost  Estimating 

•  Business  -  Financial  Management 

•  Contracting 

•  Engineering 

•  Facilities  Engineering 

•  Industrial/Contract  Property  Management 

10  This  illustration  includes  both  military  and  civilian  personnel.  Other  RAND  documents  have  used  a  similar  chart 
to  describe  just  the  civilian  DAW  in  earlier  FYs.  See,  for  example,  Gates  et  al.  (2008)  and  Gates  et  al.  (2013). 

1 1  Gates  et  al.  (2013),  pages  1-2,  and  Guo,  Hall-Partyka  and  Gates  (2014),  page  1,  show  a  very  similar  list  of  career 
fields  for  FY  2011. 
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•  Information  Technology 

•  Life  Cycle  Logistics 

•  Program  Management 

•  Production,  Quality  and  Manufacturing 

•  Purchasing 

•  Science  and  Technology  Management 

•  Test  and  Evaluation 

The  distribution  of  the  DAW  among  these  career  fields  differs  between  the  civilian  and 
military  acquisition  communities.  These  distributions  are  illustrated  in  Figures  2.2  and  2.3, 
below. 


Figure  2.2.  Civilian  DAW  by  Career  Field,  FY  201 512 


Business  -  Financial 
Mgmt 
4% 

Information 
Technology 
4% 

Test  and  Evaluation 
5% 


S&T  Manager  ^  Other 
Auditing  2°{° 


Engineering 

28% 


Facilities  Engineering 
5% 


Production,  Quality 
8t  Manuf. 

7% 


Program 

Management 


Contracting 

18% 


Life  Cycle  Logistics 
13% 


SOURCE:  Office  of  Human  Capital  Initiatives  (2015) 


12 

Other  RAND  documents  have  used  a  similar  chart  to  describe  the  civilian  DAW  in  earlier  FYs.  See,  for  example, 
Gates  et  al.  (2008),  Gates  (2009b),  and  Gates  et  al.  (2013). 
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Figure  2.3.  Military  DAW  by  Career  Field,  FY  201 513 
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SOURCE:  Office  of  Human  Capital  Initiatives  (2015) 


Approximately  two  thirds  of  civilian  acquisition  professionals  are  in  one  of  the  top  four 
career  fields:  engineering,  contracting,  lifecycle  logistics,  or  program  management.  In  the 
military  DAW,  approximately  60  percent  of  acquisition  professionals  are  in  either  the 
contracting  or  program  management  career  fields. 


Workforce  Reductions  throughout  the  1990s 

The  entire  federal  government  experienced  personnel  reductions  throughout  the  last  decade 
of  the  20th  century  (Executive  Office  of  the  President  Office  of  Management  and  Budget,  2002), 
and  the  DoD  and  DAW  were  no  exceptions  (U.S.  Department  of  Defense,  2000;  Gill,  2001; 
Office  of  the  Inspector  General,  2000).  During  the  1990s,  the  DoD  decreased  the  number  of  its 
civilian  personnel  by  approximately  37  percent,  and — aligning  with  recommendations  from  the 
Packard  Commission — the  number  of  defense  acquisition  employees  decreased  even  more  (U.S. 
Department  of  Defense,  2000;  Gill,  2001;  U.S.  President's  Blue  Ribbon  Commission  on  Defense 
Management,  1986).  These  stark  reductions  created  “imbalances  in  both  skills  and  experience 
levels  in  many  parts  of  the  DoD”  (Gates,  Eibner  and  Keating,  2006,  pp.  xiii,  1;  U.S.  Department 
of  Defense,  2000;  Office  of  the  Inspector  General,  2000). 
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Other  RAND  documents  have  used  a  similar  chart  to  describe  the  military  portion  of  the  DAW  in  earlier  FYs. 
See,  for  example,  Gates  et  al.  (2008),  Gates  (2009b),  and  Gates  et  al.  (2013). 
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The  DAWDF  and  the  Growth  Initiative 

In  the  early  2000s,  there  were  some  concerns  about  the  state  of  the  DAW.  The  DoD  civilian 
workforce  and  the  acquisition  professionals  within  that  workforce  were  aging  (U.S.  Department 
of  Defense,  2000;  Testimony  by  Dr.  Diane  M.  Disney,  2000).  A  major  report  made  for  the  Under 
Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics  (USD  (AT&L))  and  the  Under 
Secretary  of  Defense  for  Personnel  and  Readiness  (USD  (P&R))  in  2000  predicted  “a  retirement- 
driven  talent  drain”  in  the  DoD  (U.S.  Department  of  Defense,  2000,  p.  ES-1).  And,  according  to 
the  report,  “the  Office  of  the  Secretary  of  Defense  (OSD)  project[ed]  .  .  .  losses  approaching  50 
percent  in  some  key  acquisition  occupations  primarily  due  to  retirement  over  the  next  5  to  6 
years”  (U.S.  Department  of  Defense,  2000,  p.  2). 14  Additionally,  another  2000  report,  from  the 
Office  of  the  Inspector  General,  projected  significant  losses  from  the  DAW  between  the  time  of 
the  report  and  FY  2005  (Office  of  the  Inspector  General,  2000). 

There  was  also  concern  that  the  DAW  was  too  small  and  would  have  difficulty  meeting  the 
demand  for  its  services.  The  report  from  the  Office  of  the  Inspector  General  writes,  “concern  is 
warranted  because  staffing  reductions  have  clearly  outpaced  productivity  increases  and  the 
acquisition  workforce’s  capacity  to  handle  its  still  formidable  workload”  (Office  of  the  Inspector 
General,  2000,  p.  ii). 

Additionally,  the  demand  placed  on  the  defense  acquisition  personnel  began  to  grow.  The 
“DoD  Procurement  Budget”  began  to  increase  starting  in  the  late  1990s  (Gansler,  Lucyshyn  and 
Arendt,  2010,  p.  2).  And,  according  to  one  article,  the  acquisition  “workload,”  measured  by 
contract  dollars,  began  to  grow  significantly  starting  in  2001  (Anderson,  2009,  pp.  2-3).  This, 
according  to  Anderson  (2009),  “strained”  the  DAW  (p.3). 

Just  before  the  passage  of  the  DAWDF,  the  then-Chainnan  of  the  Senate  Committee  on 
Armed  Services,  Senator  Carl  Levin,  stated, 

Senior  DoD  officials  have  recognized  the  deficiencies  in  the  defense  acquisition 
workforce,  but  they  have  been  unable  to  obtain  significant  funds  that  are  needed 
to  remedy  the  problem.  ...  Our  bill  [DAWDF]  will  address  this  issue  by 
establishing  an  acquisition  workforce  development  fund  to  enable  the 
Department  of  Defense  to  increase  the  size  and  quality  of  its  acquisition 
workforce  (Levin,  2007,  p.  S 12367;  Anderson,  2009,  p.  2). 

The  DAWDF  was  part  of  the  FY  2008  NDAA  (Public  Law  110-181,  National  Defense 
Authorization  Act  for  Fiscal  Year  2008)  and  is  codified  as  10  U.S.C.  Chapter  87  Section  1705 
(Title  10  U.S.C.  Ch.  87  Section  1705).  According  to  the  legislation,  the  DoD  may  use  the  money 
in  the  DAWDF  “for  the  recruitment,  training,  and  retention  of  acquisition  personnel”  (Public 


14 

U.S.  Department  of  Defense  (2000)  cites  the  following:  “DoD  113  Report,  June  2000,  adjusted  for  seasonal 
employment;  source  ODASD  (CPP)”  (p.  2). 

A  report  from  the  Office  of  the  Secretary  of  Defense  (2000)  states  that  “Between  1999  and  2005,  as  a  result  of 
separations  and  retirements,  the  job  series  that  make  the  primary  contribution  to  the  acquisition  process  are  projected 
to  experience  cumulative  losses  ranging  from  35%  to  50%’'  (Office  of  the  Secretary  of  Defense,  2000,  p.  2-3). 
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Law  1 10-181,  National  Defense  Authorization  Act  for  Fiscal  Year  2008,  sec.  852;  Title  10 
U.S.C.  Ch.  87  Section  1705).  To  respond  to  the  workforce  shortfalls,  the  initial  emphasis  was  on 
growing  the  DAW,  through  “recruiting  and  hiring”  initiatives  (Anderson,  2009,  p.  4;  Office  of 
the  Under  Secretary  of  Defense,  2010). 

Less  than  a  year  after  the  passage  of  DAWDF,  the  FY  2009  NDAA  (Public  Law  1 10-417, 
Duncan  Hunter  National  Defense  Authorization  Act  for  Fiscal  Year  2009)  established  the 
Expedited  Hiring  Authority  within  the  DAW.  This  authority  allows  for  a  streamlined  hiring 
process,  greater  pay  flexibility,  and  bonus  payments  for  certain  DAW  positions  recognized  by 
the  Secretary  of  Defense  as  being  especially  difficult  to  fill  or  undermanned.15  Additionally,  in 
April  of  2009,  the  then  Secretary  of  Defense  Robert  Gates  announced  a  DoD-wide  acquisition 
workforce  growth  initiative.  Specifically,  Secretary  Gates  declared  a  plan  to  increase  the  size  of 
the  DAW  by  20,000  by  FY  2015  (Gates,  2009a).  Gates  planned  for  just  over  half  of  this  growth 
to  come  from  “converting”  “contractors”  into  actual  DoD  personnel  (Gates,  2009a). 

Defense  acquisition  personnel  managers  responded,  with  help  from  the  DAWDF  and  the 
Expedited  Hiring  Authority.  In  fact,  the  DoD  neared  its  DAW  growth  initiative  goals  more 
quickly  than  expected.  From  the  end  of  FY  2009  to  the  end  of  FY  2011,  the  DAW  grew  from 
approximately  133,000  personnel  to  just  under  152,000  workers — an  increase  of  over  18,000 
(Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  2015).  See  Figure  2.4  for  the  total 
DAW  workforce  size  for  FYs  2007  through  2015.  The  graphic  illustrates  that  the  DAW  grew 
most  significantly  during  FYs  2009,  2010,  and  2011. 


15  The  Expedited  Hiring  Authority  for  the  DAW  was  originally  established  in  the  FY  2009  NDAA  (Public  Law  1 10- 
417,  Duncan  Hunter  National  Defense  Authorization  Act  for  Fiscal  Year  2009,  sec.  833)  but  has  changed  slightly  as 
a  result  of  the  FY  2010  NDAA  (Public  Law  1 11-84,  National  Defense  Authorization  Act  for  Fiscal  Year  2010,  sec. 
831)  and  the  FY  2013  NDAA  (Public  Law  1 12-239,  National  Defense  Authorization  Act  for  Fiscal  Year  2013,  sec. 
803).  The  original  “Termination  date”  was  September  30,  2012  (Public  Law  1 10-417,  Duncan  Hunter  National 
Defense  Authorization  Act  for  Fiscal  Year  2009,  sec.  833),  but  the  Authorization  has  since  been  extended  to 
September  30,  2017  (Public  Law  1 12-239,  National  Defense  Authorization  Act  for  Fiscal  Year  2013,  sec.  803).  The 
implementation  of  the  Expedited  Hiring  Authority  can  be  seen  in  the  following  official  memoranda:  Office  of  the 
Secretary  of  Defense  (2008),  Office  of  the  Secretary  of  Defense  (2010),  and  Office  of  the  Secretary  of  Defense 
(2015).  These  memoranda  from  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics  and  the 
Under  Secretary  of  Defense  for  Personnel  and  Readiness  document  which  parts  of  the  DAW  (i.e.,  which  types  of 
acquisition  positions)  are  authorized  to  use  the  Expedited  Hiring  Authority.  The  Expedited  Hiring  Authority  for  the 
DAW  is  codified  as  10  U.S.C.  Chapter  87  Section  1705(g)  (Title  10  U.S.C.  Ch.  87  Section  1705(g)).  The  Authority 
(1)  allows  the  hiring  process  to  forgo  the  hiring  preference  regulations  outlined  in  5  U.S.C  Chapter  33  Sections 
3309-3318  (Title  5  U.S.C.  Ch.  33  Subchapter  I;  Title  5  U.S.C.  Ch.  33  Section  3304),  (2)  allows  for  greater  pay 
flexibility  for  “new  appointments”  (Title  5  U.S.C.  Ch.  53  Section  5333),  and  (3)  allows  for  the  use  of  “Recruitment 
and  relocation  bonuses”  (Title  5  U.S.C.  Ch.  57  Section  5753)  for  the  specified  defense  acquisition  positions. 
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Figure  2.4.  DAW  Grew  Most  Significantly  during  FYs  2009,  2010,  and  2011 
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SOURCE:  (Office  of  Human  Capital  Initiatives) 

Although  Figure  2.4  shows  that  the  total  size  of  the  DAW  began  to  grow  in  FY  2009,  the 
cohort  analysis  shows  that  cohort  sizes  began  to  increase  starting  in  FY  2008,  the  same  year  that 
the  DAWDF  was  founded.  Figure  2.5,  below,  illustrates  the  number  of  entrants  into  the  civilian 
DAW  for  FYs  2000  to  2014. 16  The  graphic  shows  that  there  is  a  significant  increase  in  the 
number  of  entrants  into  the  civilian  DAW  during  FYs  2008-2011.  This  is  a  bit  surprising,  given 
that  no  money  from  the  DAWDF  was  used  to  hire  employees  into  the  DAW  until  FY  2009 
(Office  of  the  Under  Secretary  of  Defense,  2010)  and  that  the  DAW  Expedited  Hiring  Authority 
was  not  established  until  the  FY  2009  NDAA  (Public  Law  1 10-417,  Duncan  Hunter  National 
Defense  Authorization  Act  for  Fiscal  Year  2009).  It  is  also  interesting  to  note  that  given  the 
increase  in  cohort  size  from  the  FY  2007  cohort  to  the  FY  2008  cohort,  the  total  size  of  the  DAW 
did  not  increase  during  FY  2008.  This  discrepancy  may  stem  from  the  fact  that  the  civilian  DAW 
experienced  a  greater  number  of  losses  in  FY  2008  than  in  FY  2007. 17 


16  The  methodology  used  to  create  these  cohort  numbers  is  defined  in  “Chapter  6.  Data  and  Definitions.” 

17 

Gates  et  al.  (2013)  find  in  their  analysis  that  there  are  both  more  gains  to  and  losses  from  the  civilian  DAW  in  FY 
2008  than  in  FY  2007  (p.  20). 

Additionally,  another  part  of  this  difference  may  come  from  the  way  this  study  defines  cohorts.  Specifically,  as 
outlined  in  Chapter  6,  the  cohort  numbers  only  include  “new  hires”  (Gates  et  al.,  2013,  pp.  7-8)  and  “substantive” 
transfers  (Gates  et  al.,  2013,  pp.  8-12)  into  the  civilian  DAW.  Gates  et  al.  (2013)  show  that  there  is  an  increase  in  the 
number  of  “New  hires”  from  FY  2007  to  FY  2008  (p.  20),  so  this  increase  is  captured  in  this  study’s  cohort 
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The  total  number  of  professionals  who  joined  the  civilian  DAW  during  FYs  2008-2011  was 
61,672,  with  the  largest  civilian  DAW  cohort  being  in  FY  2010,  when  over  19,000  professionals 
joined  the  civilian  DAW.  In  contrast,  a  total  of  95,916  professionals  joined  the  civilian  DAW 
during  FYs  2000-2007  or  FYs  2012-2014,  with  the  largest  of  these  cohorts  having  just  over 
10,700  entrants.18 

In  general,  besides  the  fact  that  the  increased  cohort  size  predates  the  full  implementation  of 
these  policy  initiatives,  these  findings  are  consistent  with  the  aims  of  the  DAW  growth  initiative, 
and  with  the  DAWDF  and  Expedited  Hiring  Authority  legislation,  and  suggest  that  DoD 
organizations  responded  to  these  efforts  by  bringing  more  professionals  into  the  civilian  DAW. 

The  Expedited  Hiring  Authority  was  not  established  until  the  passage  of  the  FY  2009 
NDAA;  money  from  the  DAWDF  was  not  used  to  hire  individuals  until  FY  2009;  and  then- 
Secretary  Gates  first  announced  the  DAW  growth  initiative  in  April  of  2009.  However,  cohort 
numbers  show  an  increase  (based  on  the  methodology  used  in  this  study)  starting  with  the  cohort 
in  FY  2008,  the  same  year  in  which  the  DAWDF  was  founded.  The  FY  2008-2011  cohorts  are 
hereafter  referred  to  as  the  “DAW  hiring  surge”  or  the  “DAW  growth  initiative”  cohorts. 


Figure  2.5.  Larger  Cohort  Sizes  During  DAW  Growth  Initiative,  FY2008-FY2011 
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numbers.  Chapter  6  explains  in  more  detail  this  study’s  definition  of  the  civilian  DAW  cohorts  and  explains  “new 
hires”  (Gates  et  at,  2013,  pp.  7-8)  and  “substantive”  and  “administrative”  transfers  (Gates  et  al.,  2013,  pp.  8-12). 

1 8 

Individuals  can  be  in  more  than  one  cohort  if  they  enter  the  civilian  DAW  on  more  than  one  occasion  based  on 
the  methodology  used  in  this  study  (which  is  outlined  in  Chapter  6).  In  total,  the  cohorts  consist  of  149,839  unique 
individuals;  7,439  (around  five  percent)  of  them  are  in  more  than  one  cohort.  The  sum  of  the  non-unique 
observations  for  the  FY  2000-2014  cohorts  equals  157,588. 
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Chapter  3.  Prior  DAW-Related  Studies 


The  recent  literature  on  the  DAW  stems  from  the  perceived  DoD-civilian-manpower 
dilemma  that  came  from  the  workforce  drawdowns  throughout  the  1990s.  There  were  concerns 
about  the  DAW  in  particular  but  also  about  the  wider  DoD  civilian  workforce  and  about  the 
entire  federal  workforce  (Gill,  2001;  U.S.  Department  of  Defense,  2000;  Office  of  the  Inspector 
General,  2000;  U.S.  President's  Blue  Ribbon  Commission  on  Defense  Management,  1986; 
Executive  Office  of  the  President  Office  of  Management  and  Budget,  2002;  Gates,  Eibner  and 
Keating,  2006).  In  response,  the  Under  Secretary  of  Defense  for  Acquisition,  Technology  and 
Logistics  (USD  (AT&L))  created  a  task  force  to  develop  recommended  courses  of  action  to  help 
“reshape”  the  DAW  (U.S.  Department  of  Defense,  2000,  p.  ES-1).  One  of  the  major  immediate 
recommendations  was  to  “Develop  and  Implement  Comprehensive,  Needs-based  Human 
Resource  Perfonnance  Plans  for  the  Civilian  Acquisition  Workforce”  (U.S.  Department  of 
Defense,  2000,  p.  ES-2).  The  USD  (AT&L)  and  the  Under  Secretary  of  Defense  for  Personnel 
and  Readiness  (USD  (P&R))  supported  the  task  force’s  recommendations  and  forwarded  them  to 
the  DoD  leaders,  including  the  heads  of  the  military  services  and  defense  agencies,  urging  them 
to  act  (U.S.  Department  of  Defense,  2011). 

At  the  federal  level,  President  George  W.  Bush  highlighted  the  need  for  “Strategic 
Management  of  Human  Capital”  when  he  issued  The  President’s  Management  Agenda  for  FY 
2002  (Executive  Office  of  the  President  Office  of  Management  and  Budget,  2002,  p.  1 1).  In  this 
document,  President  Bush  said  that  officials  poorly  handled  the  workforce  reductions  in  the 
1990s,  and  that  their  mismanagement  could  create  workforce  issues  during  the  following  decade. 
Bush  called  for  the  improvement  of  workforce  planning  in  the  future  throughout  the  entire 
federal  government  (Executive  Office  of  the  President  Office  of  Management  and  Budget,  2002). 

Defense  acquisition  leaders  and  federal  officials  both  called  for  strategic  workforce  plans  at 
the  beginning  of  the  21st  century,  and  these  orders  spurred  the  DoD  into  action  (Gates  et  ah, 
2008).  Within  the  Defense  Department,  acquisition  officials  were  responsible  for  creating  their 
own  workforce-management  plan  specifically  aimed  at  the  DAW  (Gates  et  ah,  2008). 

In  general,  the  first  group  of  studies  published  in  response  to  the  DoD’s  new  motivation  for 
workforce  management  focuses  on  where  defense  officials  should  start  in  thinking  about 
strategic  workforce  management.  First,  studies  from  Emmerichs,  Marcum  and  Robbert  (2004a 
and  2004b)  involve  developing  a  “methodology  for  conducting  workforce  planning” 

(Emmerichs,  Marcum  and  Robbert,  2004b,  p.  ix).  A  few  years  later,  Gates,  Eibner  and  Keating 
(2006)  completed  a  defense-sponsored  report  that  describes  how  specific  defense  organizations 
were  managing  their  workforces  and  highlights  what  the  Office  of  the  Secretary  of  Defense 
(OSD)  can  do  to  assist  in  this  planning.  Additionally,  this  study  also  points  out  the  data  resources 
available  to  defense  officials  that  will  be  useful  for  workforce  planning  (Gates,  Eibner  and 
Keating,  2006). 
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Then,  defense  officials,  acquisition  leaders,  and  researchers  focused  their  attention  on 
describing  and  understanding  the  current  state  of  the  DAW.  RAND  conducted  several  studies. 
One  RAND  study,  which  was  published  in  2009,  focuses  mainly  on  the  Navy’s  civilian 
acquisition  workforce  (Gates  et  ah,  2009).  Gates  et  al.  (2009)  use  administrative  data  to  examine 
personnel  trends  in  the  Navy’s  civilian  acquisition  workforce,  including  changes  in  the  number 
of  civilians  in  the  workforce  and  attrition  (Gates  et  al.,  2009).  Using  the  same  administrative 
data,  the  study  also  examines  the  Navy’s  internship  program  and  the  development  and  selection 
of  its  Senior  Executives  (Gates  et  al.,  2009).  The  most  notable  conclusions  yielded  from  this 
study  include  that  members  of  the  Navy  acquisition  workforce  tend  to  stay  in  the  workforce 
longer  than  do  other  Navy  civilians  (before  retirement  eligibility)  and  they  also  tend  to  be  more 
highly  educated  (Gates  et  al.,  2009). 

At  approximately  the  same  time,  a  group  of  researchers  conducted  a  similar  study,  this  time 
concentrating  on  the  entire  DAW  (Gates  et  al.,  2008).  This  study  focuses  on  many  of  the  same 
factors  examined  by  Gates  et  al.  (2009) — the  size  of  the  workforce  over  time,  education  levels, 
retirement,  attrition,  and  leadership  development  and  selection — only  this  study  was  done  from  a 
DoD-wide  perspective  (Gates  et  al.,  2008).  The  study  also  includes  unique  analyses.  Gates  et  al. 
(2008)  examine  the  size  of  and  distributions  within  the  military  portion  of  the  DAW. 
Additionally,  Gates  et  al.  (2008)  also  briefly  analyze  the  magnitude  of  the  military-to-civilian- 
DAW  transition.  The  study  finds  that  many  of  the  newly  hired  members  of  the  DAW  have  some 
type  of  prior  military  experience,  and  the  authors  illustrate  that  this  number  grew  steadily  from 
1993  to  2006  (from  around  23  percent  in  FY  1993  to  approximately  40  percent  in  FY  2006) 
(Gates  et  al.,  2008).  Gates  et  al.  (2008)  show  that  the  civilian  DAW  depends  quite  heavily  on  the 
military  ranks  to  supply  its  manpower  pipeline.  Gates  et  al.  (2013)  complete  the  same  analyses 
but  included  updated  data  (through  FY  2011).  Gates  et  al.  (2013)  also  note  that  they  made  some 
subtle  changes  in  the  way  they  interpreted  some  of  the  data,  although  they  report  that  these 
changes  did  not  significantly  affect  the  final  results  (Gates  et  al.,  2013). 

Using  the  same  administrative  data  from  the  DoD,  Guo,  Hall-Partyka  and  Gates  (2014)  took 
another  step  forward  in  understanding  the  DAW:  they  uncovered  relationships  between  various 
personnel  factors — such  as  “education  and  performance”  (pp.  xi,  3,13)  — and  retention,  and  they 
analyzed  “promotion  patterns”  (p.  iii)  within  the  DAW.  Their  analysis  yielded  some  interesting 
results.  First,  Guo,  Hall-Partyka  and  Gates  (2014)  show  that  acquisition  professionals  who 
complete  an  education  upgrade  tend  to  remain  in  the  DAW  for  a  longer  period  than  those  who  do 
not.  Second,  Guo,  Hall-Partyka  and  Gates  (2014)  find  that  higher  perfonning  civilian  defense 
acquisition  professionals  tend  to  have  lower  retention  levels  than  those  whose  performance 
ratings  are  worse,  and  the  authors  point  out  that  this  is  especially  true  “for  employees  who 
entered  the  [civilian  DAW]  at  more  senior  grades”  (p.  xi).  Guo,  Hall-Partyka  and  Gates  (2014) 
find  that  individuals  indicated  as  “Veterans”  tend  to  have  greater  retention  in  the  civilian  DAW 
than  individuals  not  indicated  as  such  (pp.  14,  21).  The  authors  also  find  that,  all  else  equal, 
“Employees  who  transferred  into  the  [civilian  DAW]”  from  a  non-DAW  DoD  civilian  position 
have  greater  retention  than  employees  who  came  from  other  sources  (Guo,  Hall-Partyka  and 
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Gates,  2014,  pp.  14,  22).  Lastly,  Guo,  Hall-Partyka  and  Gates  (2014)  highlight  that  demographic 
factors  do  not  have  a  significant  relationship  with  promotion  to  the  SES  ranks.  Rather,  it  is  the 
organizational  background  of  defense  acquisition  personnel  that  has  predictive  power  in  regard 
to  whether  someone  is  promoted  to  be  a  Senior  Executive  (Guo,  Hall-Partyka  and  Gates,  2014). 

This  dissertation  expands  on  the  existing  body  of  RAND  DAW  research  (Gates  et  ah,  2008; 
Gates  et  al.,  2009;  Gates  et  ah,  2013;  Guo,  Hall-Partyka  and  Gates,  2014).  This  dissertation  is 
unique  in  that  it  develops  a  methodology  for  reaching  back  into  civilian  DAW  entrants’  pasts  to 
characterize  their  prior  work  experience  and  uses  this  characterization  technique  to  understand 
how  policy  affects  the  workforce.  Specifically,  this  analysis  uses  this  characterization  technique 
to  (1)  explore  how  the  recent  growth  initiative  has  changed  who  the  civilian  DAW  has  brought 
into  its  ranks,  (2)  understand  how  different  past  work  experiences  relate  to  retention  outcomes, 
and  (3)  estimate  how  the  growth  initiative  has  influenced  overall  cohort-level  retention  rates. 
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Chapter  4.  Prior  Work  Experience,  Workforce  Socialization,  and 
Retention 


One  way  to  characterize  workforce  entrants  is  by  their  prior  work  experience.  There  are  two 
main  reasons  why  it  makes  sense  to  start  by  examining  prior  work  experience.  First,  the 
workforce  socialization  theory  and  empirical  analyses  argue  that  prior  experiences  play  an 
important  role  in  an  individual’s  assimilation  into  a  new  organization  and  that  the  assimilation 
process  influences  retention.  Secondly,  this  is  a  natural  place  to  start  given  the  important  role 
prior  work  experience  plays  in  the  employee  selection  process.  Usually,  past  work  experience  is 
a  large  part  of  each  applicant’s  resume  or  job  application  (Dokko,  Wilk  and  Rothbard,  2009; 
Adkins,  1995).  These  prior  experiences  signal  what  “skills”  and  “habits”  each  applicant  has 
(Dokko,  Wilk  and  Rothbard,  2009,  p.  52). 

The  literature  refers  to  an  individual’s  process  of  assimilating  into  a  new  organization  or 
workforce  as  “organizational  socialization”  (Bauer  et  ah,  2007,  p.  7;  Saks  and  Ashforth,  1997,  p. 
234;  Adkins,  1995,  p.  839;  Carr  et  al.,  2006,  p.  343;  Van  Maanen  and  Shein,  1977,  p.  1).  Bauer  et 
al.  (2007)  define  organizational  socialization  as  “the  process  by  which  newcomers  make  the 
transition  from  being  organizational  outsiders  to  being  insiders”  (p.  707).  Van  Maanen  and 
Shein  (1977)  write, 

At  heart,  organizational  socialization  is  a  jejune  phrase  used  by  social  scientists 
to  refer  to  the  process  by  which  one  is  taught  and  leams  "the  ropes"  of  a 
particular  organizational  role.  In  its  most  general  sense,  organizational 
socialization  is  then  the  process  by  which  an  individual  acquires  the  social 
knowledge  and  skills  necessary  to  assume  an  organizational  role  (p.  3). 

In  the  past,  researchers  thought  about  and  modeled  the  socialization  process  in  a  number  of 
ways  (Saks  and  Ashforth,  1997).  In  1997,  Saks  and  Ashforth  (1997)  integrated  the  separate 
theories  into  an  all-inclusive  organizational-socialization  model,  which  they  call  the  “Multi- 
Level  Process  Model  of  Organizational  Socialization”  (pp.  238-241).  Other  researchers  have 
used  similar  versions  of  this  integrated  approach  to  guide  analyses  aimed  at  understanding  what 
affects  individuals’  assimilation  into  a  new  organization  or  workforce  and  what  influences  all  of 
this  may  have  on  workforce  outcomes  (Bauer  et  al.,  2007;  Carr  et  al.,  2006;  Kammeyer-Mueller 
and  Wanberg,  2003).  A  simplified  version  of  this  model  is  illustrated  in  Figure  4.1,  below. 

As  illustrated  in  Figure  4.1,  one  can  think  of  the  entire  socialization  process  in  the  logic 
model  format,  consisting  of  inputs,  a  process,  outputs,  and  outcomes.  The  inputs  are  the 
newcomers  (i.e.,  entrants)  and  any  introductory  courses  that  the  organization  provides  (Saks  and 
Ashforth,  1997;  Bauer  et  al.,  2007;  Kammeyer-Mueller  and  Wanberg,  2003).  The  process  is 
simply  assimilating  into  the  new  workforce,  with  the  goals  (i.e.,  the  outputs)  being  “Social 
integration”  in  the  new  organization,  “Skill  acquisition,”  developing  “Role  clarity,”  and 
understanding  “Person-job/organization  fit”  (Saks  and  Ashforth,  1997,  p.  239;  Kammeyer- 
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Mueller  and  Wanberg,  2003;  Bauer  et  al.,  2007).  Finally,  the  model  predicts  that  effective 
workforce  socialization  should  produce  important  workforce  outcomes,  including  “Higher  job 
satisfaction,”  “Higher  organizational]  commitment,”  “Higher  performance,”  and  “Lower 
absenteeism  &  turnover”  (Saks  and  Ashforth,  1997,  p.  239;  Bauer  et  ah,  2007;  Kammeyer- 
Mueller  and  Wanberg,  2003).  The  Saks  and  Ashforth  (1997)  model  highlights  that,  among  other 
things,  “individuaf’-specific  characteristics,  such  as  “previous  work  experience,” 

“Demographic [s],”  and  “Personality,”  influence  how  effectively  and  easily  an  individual 
assimilates  into  his  or  her  new  environment,  which — in  turn — will  influence  outcomes  such  as 
retention  (Saks  and  Ashforth,  1997,  pp.  238-241). 


Figure  4.1.  A  Simplified  Version  of  Saks  and  Ashforth’s  (1997)  Multi-Level  Process  Model  of 

Organizational  Socialization 


SOURCE:  Saks  and  Ashforth  (1997),  pp.  238-241 

A  number  of  studies  have  looked  into  the  relationship  between  prior  work  experience  and 
workforce  outcomes. 
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Carr  et  al.  (2006)  examine  the  relationship  between  “prior  occupational  experience”  and  job 
retention  (in  the  same  general  occupation)  (p.  343).  The  authors  surveyed  more  than  200  truck 
driver  new  hires  in  a  company,  gathering  self-reported  infonnation  on  their  prior  experience  as  a 
driver.  In  addition,  the  surveys  collected  infonnation  on  the  new  hires’  job  expectations,  how 
closely  their  values  fit  with  the  values  of  the  company,  and  how  well  they  believe  their  skills  and 
abilities  match  their  new  job  requirements.  Carr  et  al.  (2006)  found  that  prior  occupational 
experience  had  a  significant  relationship  with  retention,  but  they  found  that  other  factors — such 
as  an  individual’s  perceived  “person-job  (P-J)  fit”  and  “value  congruence” — “mediated”  this 
relationship  (p.  343). 

Kammeyer-Mueller  and  Wanberg  (2003)  investigate  how  a  number  of  different  factors,  such 
as  “Preentry  knowledge,”  “personality,”  and  “Socializing  Influences”  from  “leaders”  (p.  785), 
may  relate  to  workforce  outcomes,  including  retention.  The  authors  gather  self-reported  survey 
data  on  “white-collar  workers”  (Kammeyer-Mueller  and  Wanberg,  2003,  p.  791)  from  a  number 
of  different  industries  (Kammeyer-Mueller  and  Wanberg,  2003).  More  than  800  participants 
reported  information  regarding,  among  other  things,  “how  much  infonnation  [they  had]  about 
their  new  jobs  in  advance,”  how  “proactive”  their  personality  is,  and  how  much  influence  leaders 
had  in  their  assimilation  into  the  new  workforce  (Kammeyer-Mueller  and  Wanberg,  2003,  p. 

785).  To  measure  retention,  Kammeyer-Mueller  and  Wanberg  (2003)  collect  “employment”  (p. 
785)  data  from  the  participants’  companies.  Using  a  Cox  Proportional  Hazard  model,  the  authors 
find  that  “Preentry  knowledge”  and  “Leader  influence”  are  related  to  higher  rates  of  retention 
(Kammeyer-Mueller  and  Wanberg,  2003,  p.  790). 

Beyer  and  Hannah  (2002)  use  “qualitative”  methods  to  investigate  how  prior  work 
experience  may  influence  an  individual’s  assimilation  into  a  new  workforce  (p.  636).  To  do  this, 
the  authors  focus  on  a  company  called  SEMATECH,  which  is  a  research  “consortium”  that  uses 
loaned  employees  from  other  companies  in  the  industry  (Beyer  and  Hannah,  2002,  p.  638).  Beyer 
and  Hannah  (2002)  closely  follow  14  individuals  who  are  sent  (i.e.,  loaned)  to  work  at 
SEMATECH  by  their  employer.  The  authors  gathered  information  about  each  individual’s  prior 
work  experience  and  conducted  a  number  of  interviews  with  each  of  the  14  employees  regarding 
their  particular  socialization  process.  In  the  end,  the  authors  found  that  prior  work  experience 
influenced  the  individuals’  socialization  process  through  three  mechanisms:  “through  the 
personal  identities  they  had  developed  and  carried  with  them,  through  the  know-how  they  had 
acquired  in  past  jobs  and  how  well  it  fit  with  their  new  jobs,  and  through  the  personal  tactics  they 
had  learned  for  managing  their  work  and  managing  change”  (Beyer  and  Hannah,  2002,  p.  636). 
The  authors  discovered  that  the  individuals  with  the  more  “diverse”19  professional  backgrounds 
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Beyer  and  Hannah  (2002)  do  not  explicitly  define  what  they  mean  by  “diversity” — whether  they  mean  job 
diversity,  organizational  diversity,  or  both.  However,  they  write  “The  diversity  of  individuals’  work  experience  is 
thus  an  indicator  of  the  practice  they  have  had  in  the  past  in  adjusting  to  new  settings  and  jobs”  (Beyer  and  Hannah, 
2002,  pp.  641-642).  Based  on  this  statement,  Beyer  and  Hannah  (2002)  seem  to  have  a  broad  definition  of 
“diversity”  that  includes  both  job  and  organizational  diversity  and  does  not  distinguish  between  the  two.  The  authors 
list  examples  of  “Very  Diverse,”  “Diverse,”  and  “Narrow”  “Past  Work  Experience”  on  page  641  (Beyer  and 
Hannah,  2002,  p.  641). 
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tended  to  assimilate  into  the  new  organization  more  successfully  (Beyer  and  Hannah,  2002,  p. 
636). 

A  study  from  Dokko,  Wilk  and  Rothbard  (2009)  investigates  the  relationship  between  prior 
work  experience  and  a  different  workforce  outcome — performance.  The  authors  gather  the  prior 
career  infonnation  for  employees  in  a  call  center  and  use  regression  techniques  to  understand 
how  this  past  work  experience  relates  to  performance  ratings.  The  call  center  employees  receive 
ratings  on  their  “knowledge  and  skill”  and  on  their  overall  perfonnance,  allowing  the  authors  to 
examine  how  (i.e.,  through  what  mechanisms)  prior  experience  influences  performance  (Dokko, 
Wilk  and  Rothbard,  2009,  pp.  52-56).  The  results  show  that  “prior  experience”  is  related  to 
higher  levels  of  “knowledge  and  skill,”  which  is  related  to  higher  perfonnance  ratings  (Dokko, 
Wilk  and  Rothbard,  2009,  p.  63).  However,  the  authors  discover  that  once  they  control  for  the 
differences  in  “knowledge  and  skill,”  prior  work  experience  is  actually  negatively  related  to 
performance  outcomes  (Dokko,  Wilk  and  Rothbard,  2009,  p.  63). 20  One  explanation  for  this  is 
that  prior  work  experience  may  improve  an  individual’s  perfonnance  through  expertise  gained 
during  that  past  experience  but  may  reduce  an  individual’s  ability  to  adapt  to  the  “habits”  or 
customs  of  the  new  organization  (Dokko,  Wilk  and  Rothbard,  2009,  pp.  52,  62-64).  The  authors 
found  the  negative  aspects  of  prior  experience,  i.e.,  the  “habits”  that  experienced  newcomers 
bring  with  them,  to  be  longer  lasting  than  the  positive  benefits,  i.e.,  increased  “knowledge  and 
skill”  (Dokko,  Wilk  and  Rothbard,  2009,  p.  64). 

In  the  context  of  the  civilian  DAW,  this  literature  outlines  several  mechanisms  through  which 
prior  work  experience  may  influence  the  socialization  into  and  the  propensity  to  remain  in  the 
civilian  DAW.  As  described  above,  Carr  et  al.  (2006)  find  that  entrants  with  “prior  occupational 
work  experience”  tend  to  have  higher  rates  of  retention  than  those  without  that  experience  (p. 
343).  Accordingly,  one  would  expect  entrants  with  prior  acquisition  experience  to  have  higher 
levels  of  retention  in  the  civilian  DAW  than  those  without.  On  the  other  hand,  Dokko,  Wilk  and 
Rothbard  (2009)  highlight  the  important  role  that  organizational  “habits”  play  in  workforce 
outcomes  (pp.  52,  62-64).  Dokko,  Wilk  and  Rothbard  (2009)  focus  on  performance  outcomes, 
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but  insofar  as  the  workforce  outcomes  (performance  and  retention)  move  in  the  same  direction," 
one  would  also  expect  entrants  with  prior  work  experience  in  the  DoD  to  more  easily  assimilate 
into  the  DoD  civilian  work  environment  and  therefore  have  greater  levels  of  civilian  DAW 
retention.  Beyer  and  Hannah  (2002)  find  that  the  variety  and  “diversity”  of  prior  work 
experience  plays  an  important  role  in  how  entrants  assimilate  into  a  new  workforce  (p.  647). 
Based  on  this  study  (Beyer  and  Hannah,  2002),  one  would  conclude  that  entrants  with  a  wide 


20 

An  earlier  study,  Adkins  (1995),  examining  “mental  health  specialists”  (p.  842),  also  finds  some  evidence  that 
“previous  [related]  work  experience”  may  actually  hinder  workforce  “socialization”  (pp.  855-856). 
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Saks  and  Ashforth  (1997)  theorize,  in  their  “Multi-Level  Process  Model  of  Organizational  Socialization,”  that 
successful  workforce  assimilation  will  lead  to  both  higher  levels  of  performance  and  retention  (pp.  238-241). 
However,  Guo,  Hall-Partyka  and  Gates  (2014)  find  evidence  that  these  two  workforce  outcomes  may  move  in 
opposite  directions  in  the  civilian  DAW;  they  find  that  civilian  defense  acquisition  professionals  with  higher 
performance  ratings  tend  to  have  lower  rates  of  retention. 
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variety  of  experiences  would  more  easily  adapt  to  their  new  work  environment  and  remain  in  the 
civilian  DAW  for  a  longer  period  of  time. 

There  is  a  developed  literature  on  the  socialization  process,  and  this  research  shows  that  prior 
work  experience  may  influence  workforce  assimilation  and  thereby  affect  workforce  outcomes, 
such  as  retention.  Additionally,  from  a  practical  perspective,  prior  work  experience  typically 
plays  an  important  role  in  the  recruitment,  assessment,  and  selection  of  new  employees.  For 
these  reasons,  this  study  characterizes  the  civilian  DAW  entrants  by  their  prior  work  experience. 
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Chapter  5.  Approach 


The  goals  of  this  study  are  to  (1)  understand  how  the  DAW  growth  initiative  changed 
workforce  characteristics,  focusing  on  prior  work  experience,  and  (2)  evaluate  how  these 
changes  may  influence  workforce  retention  outcomes.  To  do  this,  this  study  asks  three  research 
questions  and  uses  descriptive-  and  retention-analysis  techniques  to  address  them  (see  Table  5.1). 
The  first  step  involves  developing  a  methodology  for  characterizing  entrants’  prior  work 
experience  and  then  using  this  characterization  to  describe  cohorts  before  and  during  the  growth 
initiative  (the  descriptive  analysis).  Subsequently,  after  adding  retention  information  and  other 
control  factors  to  the  analysis  data  files,  survival  analysis  techniques  are  used  to  understand  how 
prior  work  experience  relates  to  retention  in  the  civilian  DAW  and  to  estimate  how  the  growth 
initiative  has  influenced  overall  cohort-level  retention. 

Table  5.1.  Research  Questions  and  Methods 


Research  Question 

Method 

1.  How  did  the  growth  initiative  change  who  the 
civilian  DAW  hired  into  its  ranks,  in  terms  of 
prior  work  experience? 

Descriptive  analysis 

2.  How  does  entrants’  past  work  experience 
relate  to  civilian  DAW  retention? 

Retention  analysis  (Part  1) 

3.  What  does  this  suggest  in  terms  of  how  the 
growth  initiative  may  have  influenced  overall 
cohort-level  retention  in  the  civilian  DAW? 

Retention  analysis  (Part  II) 

Descriptive  Analysis:  How  Did  the  Growth  Initiative  Change  Who  the 
Civilian  DAW  Hired,  in  Terms  of  Prior  Work  Experience? 

The  descriptive  analysis  (1)  defines  the  civilian  DAW  cohorts  from  FYs  2000-2014;  (2) 
characterizes  each  entrant’s  prior  work  experience;  and  then  (3)  uses  this  characterization  to 
examine  how  cohorts  from  before  the  hiring  surge  (prior  to  FY  2008)  differ  from  those  during 
the  surge  (FYs  2008-2011). 

As  outlined  in  Chapter  2,  the  growth  initiative  was  an  ambitious  surge  in  hiring,  and  some  of 
the  surge  was  aimed  at  hiring  contractors  into  government  positions  (Gates,  2009a).  Therefore, 
given  the  parameters  of  the  initiative,  one  would  expect  the  hiring  surge  cohorts  to  have  a  greater 
percentage  of  entrants  who  came  from  positions  not  officially  within  the  DoD.  Moreover,  one 
would  logically  hypothesize  that  the  growth  initiative  cohorts  would  also  have  a  higher  fraction 
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of  entrants  with  no  prior  DoD  experience  compared  to  the  cohorts  prior  to  the  growth  initiative 
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years.-  The  research  question  addressed  in  the  descriptive  analysis  and  the  hypothesis  tested  are 
shown  in  Table  5.2,  below. 


Table  5.2.  Descriptive  Analysis:  Question  and  Hypothesis 


Question 

Hypothesis 

Research  Question  1.  How  did  the  growth  initiative 
change  who  the  civilian  DAW  hired  into  its  ranks,  in  terms 
of  prior  work  experience? 

The  growth  initiative  cohorts  have  a  greater  percentage  of 
entrants  who  joined  from  outside  of  the  DoD  and  a 
greater  percentage  of  entrants  with  no  prior  DoD 
experience  at  all  compared  to  the  cohorts  prior  to  the 
hiring  surge. 

Retention  Analysis  Part  I:  How  Does  Prior  Work  Experience  in  the  DoD 
Relate  to  Retention  in  the  Civilian  DAW? 

The  first  part  of  the  retention  analysis  builds  from  the  dataset  created  for  the  descriptive  tests, 
adding  tenure  and  loss-type  information,  along  with  a  number  of  control  factors,  to  investigate — 
using  survival  analysis  techniques — how  different  past  work  experiences  relate  to  retention 
trends  in  the  civilian  DAW.  This  analysis  (Retention  Analysis  Part  I)  focuses  on  cohorts  prior  to 
the  growth  initiative  (i.e.,  prior  to  the  FY  2008  cohort)  because  this  allows  the  survival  tests  to 
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capture  longer-term  retention." 

It  is  not  entirely  clear  how  prior  DoD  work  experience,  as  defined  in  this  study,  should  relate 
to  retention  outcomes.  First,  this  study  has  no  explicit  past-work-experience  infonnation  on 
entrants  who  join  the  civilian  DAW  with  no  prior  DoD  experience.  These  entrants  may  be 
experienced  contractors  who  are  joining  the  civilian  DAW  with  years  of  defense  acquisition 
experience,  or  they  could  be  new  hires  with  no  acquisition  experience  whatsoever.  One  would 
expect  these  two  types  of  non-DoD-experienced  entrants  to  have  different  retention  patterns 
within  the  civilian  DAW.  Additionally,  DoD  experience  is  a  broad  term — different  DoD 
employment  histories  could  relate  to  retention  differently.  At  the  same  time,  as  outlined  in  the 
previous  chapter,  prior  work  experience  can  influence  workforce  outcomes  through  a  number  of 
different  mechanisms. 

This  analysis  uses  Cox  Proportional  Hazard  models  to  test  for  differences  in  retention  trends 
across  different  past-work-experience  groups  and  uses  control  techniques  to  account  for 
confounding  factors.  The  analyses  control  for  demographic  factors  and  factors  regarding  the 
entrants’  civilian  DAW  entry  positions,  among  other  things.  Career  level  and  compensation  are 
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“Chapter  6.  Data  and  Definitions”  outlines  how  this  study  defines  prior  DoD  experience.  In  short,  individuals  who 
appear  in  an  end-of-FY  ADMF  or  DoD  civilian  personnel  file  (starting  with  FY  1980)  are  indicated  as  having  prior 
DoD  experience.  Only  official  DoD  employees  are  included  in  the  personnel  files;  defense  contractors  are  not 
included. 

93 

This  analysis  creates  and  utilizes  retention  data  through  FY  2014. 
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two  of  the  important  factors  included  as  controls  in  the  Cox  Proportional  Hazard  models. 
Assuming  that  civilian  DAW  entrants  with  prior  acquisition  experience  are  more  likely  to  join 
the  civilian  DAW  at  higher  career  levels  and  have  higher  levels  of  compensation,  these  factors 
are  effective  proxies  for  pre-entry  acquisition  “knowledge  and  skill.”24  With  this  assumption,  the 
models  used  in  this  analysis  test  for  differences  in  retention  trends  across  past-work-experience 
groups,  controlling  for  “knowledge  and  skill”  (among  other  things).  Therefore,  although  it  is  not 
entirely  clear,  one  logical  hypothesis  given  the  findings  of  Dokko,  Wilk  and  Rothbard  (2009)  is 
that  entrants  with  prior  work  experience  in  a  DoD  organization  would  more  easily  assimilate  into 
the  civilian  DAW  and,  therefore,  tend  to  stay  for  a  longer  period  of  time,  all  else  equal.  Using 
this  same  logic,  one  would  hypothesize  that  entrants  with  prior  experience  in  the  civilian  DAW 
or  in  the  wider  DoD  civilian  workforce  would  most  easily  fit  into  their  civilian  DAW  work 
environment  and  therefore  have  greater  retention  than  entrants  without  prior  experience  in  the 
DoD.  Table  5.3,  below,  lists  the  specific  question  asked  in  this  analysis  and  outlines  the  proposed 
hypothesis. 


Table  5.3.  Retention  Analysis  Part  I:  Question  and  Hypothesis 


Question 

Hypothesis 

Research  Question  2.  How  does  entrants’  past  work 
experience  relate  to  civilian  DAW  retention? 

Entrants  with  prior  DoD  experience  have  higher 
retention  in  the  civilian  DAW  than  those  with  no  prior 

DoD  experience.  Entrants  with  prior  experience  in  the 
civilian  DAW  or  in  the  wider  DoD  civilian  workforce  have 
higher  civilian  DAW  retention  than  entrants  with  no  prior 
DoD  experience. 

Retention  Analysis  Part  II:  How  Did  the  Growth  Initiative  Influence 
Retention  in  the  Civilian  DAW? 

Part  II  of  the  retention  analysis  examines  how  the  shift  in  cohort  characteristics  brought  about 
by  the  growth  initiative  relates  to  retention  trends  and  investigates  what  role  the  shift  in  past- 
work-experience  characteristics,  specifically,  plays  in  this  relationship.  This  analysis  asks  two 
specific  questions  to  explore  these  issues.  First,  it  asks  what  retention  trends  would  have  been  for 
those  hired  during  the  growth  initiative  (the  “policy  cohorts”)  if  their  characteristics  (including 
prior-work-experience  characteristics)  had  resembled  those  who  were  hired  before  the  growth 
initiative  (the  “pre-policy  cohorts”).  Second,  for  the  sake  of  robustness,  the  analysis  also  asks 
what  retention  trends  would  have  been  for  those  who  were  hired  before  the  growth  initiative  if 
their  characteristics  had  more  closely  resembled  those  who  were  hired  during  the  growth 
initiative. 

Assuming  that  the  hypotheses  for  the  descriptive  analysis  and  Part  I  of  the  retention  analysis 
hold — that  the  growth  initiative  cohorts  have  a  greater  percentage  of  non-DoD-experienced 

24 

The  phrase  “knowledge  and  skill"  comes  from  Dokko,  Wilk  and  Rothbard  (2009).  See,  for  example,  pages  52-56, 
63,  and  64  in  Dokko,  Wilk  and  Rothbard  (2009). 
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entrants,  and  that  these  entrants  tend  to  have  lower  retention  rates  relative  to  entrants  with  DoD 
experience — one  would  hypothesize  that  the  growth  initiative  would  produce  cohorts  with  lower 
overall  rates  of  retention  (compared  to  what  would  have  happened  if  pre-growth  initiative  trends 
had  continued)  and  that  the  shift  in  past-work-experience  characteristics  would  play  an  important 
role  in  this  difference.  Table  5.4,  below,  outlines  the  specific  test  questions  addressed  in  Part  II 
of  the  retention  analysis  and  lists  their  respective  hypotheses. 


Table  5.4.  Retention  Analysis  Part  II:  Questions  and  Hypotheses 


Question 


Hypothesis 


Research  Question  3.  How  did  the  growth  initiative  influence 
retention  in  the  civilian  DAW? 

(a)  What  would  retention  trends  have  been  for  the  policy 
cohorts  if  the  individuals  in  these  cohorts  had  looked 
more  like  those  in  the  pre-policy  cohorts? 

(b)  What  would  retention  trends  have  been  for  the  pre¬ 
policy  cohorts  if  the  individuals  in  these  cohorts  had 
looked  more  like  those  in  the  policy  cohorts? 


The  growth  initiative  produced  cohorts  with  lower 
civilian  DAW  retention. 

(a)  If  the  policy  cohorts  had  looked  more  like  pre¬ 
policy  cohorts,  retention  would  have  been  higher. 

(b)  If  the  pre-policy  cohorts  had  looked  more  like 
the  policy  cohorts,  retention  would  have  been 
lower. 


In  order  to  address  the  questions  listed  in  Table  5.4,  this  analysis  uses  a  synthetic-cohort- 
comparison  technique.  This  technique  involves  using  propensity  score  weighting  to  create  a 
synthetic  cohort,  i.e.,  a  counterfactual,  to  estimate  what  the  retention  trends  would  have  been  if 
the  cohort  characteristics  had  been  different.  Specifically,  to  address  question  (a)  in  Table  5.4, 
this  study  weights  a  policy  cohort  group  to  look  like  the  pre-policy  cohorts,  creating  a  synthetic 
pre-policy  cohort  group,  and  compares  the  retention  trends  for  this  synthetic  pre-policy  cohort 
group  to  the  retention  trends  for  the  unweighted  policy  cohort  group.  This  study  uses  the  same 
technique  but  in  the  other  direction  to  address  question  (b)  in  Table  5.4:  It  weights  a  pre-policy 
cohort  group  to  look  more  like  the  policy  cohorts,  creating  a  synthetic  policy  cohort  group,  and 
compares  the  retention  trends  for  this  synthetic  policy  cohort  group  to  the  retention  trends  for  the 
unweighted  pre-policy  cohort  group.  This  analysis  uses  Cox  Proportional  Hazard  models  to 
compare  retention  trends. 

One  major  advantage  of  the  synthetic-cohort  technique,  which  compares  weighted  and 
unweighted  versions  of  the  same  dataset,  is  that  it  effectively  controls  for  any  global  time- 
specific  confounding  factors,  such  as  the  strength  of  the  national  economy  and  labor  market. 
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Chapter  6.  Data  and  Definitions 


In  order  to  address  the  questions  described  above,  this  study  utilizes  annual  defense 
personnel  data  files  maintained  by  the  Defense  Manpower  Data  Center  (DMDC).  First,  the 
analysis  uses  these  data  to  define  and  create  the  civilian  DAW  cohorts  (FY  cohorts  2000-2014) 
and  characterize  each  entrant’s  prior  work  experience  for  the  descriptive  analysis.  After  that, 
additional  information,  including  retention  data  and  control  factors,  are  added  to  the  data  file  for 
the  pre-policy  and  policy  cohorts,  and  these  comprehensive  files  are  used  in  the  retention 
analyses.  These  data  and  definitions  are  described  below. 


Raw  Data  Files 

The  datasets  used  for  the  descriptive  and  retention  analyses  are  a  combination  and  distillation 
of  the  DoD  civilian,  active-duty  military,  and  acquisition  workforce  data  maintained  by  the 
DMDC.  When  linked  together,  these  administrative  files  show  which  individuals  enter  the 
civilian  DAW  and  provide  the  information  necessary  to  characterize  each  entrant’s  past  DoD 
work  experience,  including  their  prior  acquisition  workforce  experience.  Past  DAW  analyses  use 
similar  data  files  (see  Gates  et  al.  (2008),  Gates  et  al.  (2013),  and  Guo,  Hall-Partyka  and  Gates 
(2014)).  The  specific  data  files  used  in  this  research  and  the  information  contained  in  each  of 
them  are  described  below. 

•  Annual  DoD  Civilian  Personnel  Inventory  Files 

The  DoD  civilian  personnel  files  that  are  maintained  at  DMDC  are  snapshots  of  the 
DoD  civilian  personnel  inventory.  These  files  contain  all  Appropriated  Fund  (APF) 
Civilians — or  all  “APF  direct  hire  civilians  paid  exclusively  from  DoD  APFs”  (U.S. 
Department  of  Defense,  2013,  p.  1),  except  for  certain  intelligence  personnel  as  outlined 
in  DoDI  1444.02  (U.S.  Department  of  Defense,  2013).25  The  records  include  a  vast 
number  of  variables  for  each  individual  included  in  the  data  files,  including  person  and 
workforce-position  descriptors.26 

This  study  only  uses  the  annual  end-of-FY  files  and  has  been  granted  access  to  a 
number  of  these  variables,  including  both  position  and  person  descriptors.  For  example, 
the  available  position  infonnation  describes  each  individual’s  position  within  the  DoD, 

25 

The  DoD  civilian  files  contain  all  types  of  workers,  including  full-time,  part-time,  and  intermittent  employees. 
This  includes  both  non-seasonal  and  seasonal  employees.  Intermittent  employees  are  those  individuals  who  do  not 
have  a  regular  set  work  schedule  ("DoD  APF  Civ  Personnel  Edit  Unpacked  File  (200803  and  After,  Data 
Dictionary),"  2010).  Flowever,  for  some  reason,  the  end-of-FY  1980  and  end-of-FY  1981  files  report  no  seasonal 
employees.  It  is  not  known  if  the  1980  and  1981  files  contain  no  seasonal  employees  or  if  seasonal  employees  in 
these  files  were  not  properly  recorded. 

26 

See  DoDI  1 444.02  Data  Submission  Requirements  for  DoD  Civilian  Personnel:  Appropriated  Fund  (APF) 
Civilians  (U.S.  Department  of  Defense,  2013).  DoDI  1444.02  outlines  which  individuals  are  included  in  the  DoD 
civilian  personnel  files  that  are  used  in  this  analysis  and  defines  what  other  information  is  included  for  these 
individuals  (U.S.  Department  of  Defense,  2013). 
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including  which  defense  organization  the  individual  works  in  and  which  specific  job  and 
grade  he  or  she  holds.  The  available  data  also  provide  individual-level  demographic 
information,  including — among  other  things — age  and  gender  descriptors.  Additionally, 
these  files  contain  supervisor  performance  ratings  for  each  individual.  This  research  uses 
DoD  Civilian  files  starting  in  FY  1980. 

•  Annual  Active  Duty  Master  Files  (ADMF)27 

DMDC  maintains  “a  centralized  database  of  active  duty  uniformed  personnel”  (U.S. 
Department  of  Defense,  2015b,  p.  1).  This  database  includes  information  on  individuals 
from  all  of  the  military  services,  the  Coast  Guard,”  and  the  other  uniformed  services, 
including  Public  Health  Services  and  the  National  Oceanic  and  Atmospheric 
Administration.  These  active  duty  personnel  files  from  DMDC  provide  snapshots  of  the 
inventory  of  all  of  these  active  duty  unifonned  personnel.  The  data  contain  a  number  of 
variables,  including  personal  demographic  and  workforce  position  descriptors.29 

This  study  uses  subsections  of  the  larger  uniformed  personnel  data  files  that  are 
maintained  by  DMDC  and  only  uses  the  annual  end-of-FY  records.  Specifically,  the  data 
files  used  in  this  analysis  consist  only  of  active-duty  military  (Anny,  Navy,  Marine 
Corps,  Air  Force)  and  Coast  Guard30  personnel  who  are  included  in  the  Active 
Component  End  Strength  calculations,  per  DoDI  1 120.1 1  (U.S.  Department  of  Defense, 
2015a).  However,  Cadets  and  Midshipmen  are  excluded.  This  study  has  been  given 
access  to  a  handful  of  variables  from  these  files,  including  personal  descriptors,  such  as 
education  level  and  gender,  and  position  information,  such  as  occupation  codes.  This 
research  uses  ADMFs  starting  in  FY  1980. 

•  Annual  DAWIA  Data  Files 

The  DAWIA  data  files  were  created  as  a  response  to  the  DAWIA,  which  mandated 
the  DoD  to  track  its  acquisition  workforce.  Hence,  the  earliest  version  of  these  data  files 
is  from  1992.  The  DAWIA  data  maintained  at  DMDC  provide  snapshots  of  official 
defense  acquisition  positions  and  personnel.  Specifically,  they  record  which  DoD 
positions  and  personnel  are  a  part  of  the  DAW  and  mark  which  acquisition  career  field 
each  falls  under.  Additionally,  these  data  files  include  a  number  of  other  variables  that 

3 1 

describe  these  individuals  and  positions. 


“7  The  previous  DAW  analyses  (Gates  et  al.  (2008),  Gates  et  al.  (2013),  and  Guo,  Hall-Partyka  and  Gates  (2014)) 
have  used  what  is  called  the  Work  Experience  Files  (WEX)  for  their  active  duty  information.  The  WEX  is  the  active 
duty  military  personnel  transaction  file.  It  is  derived  from  the  ADMFs  and  captures  changes  in  the  individuals’ 
records.  This  analysis  uses  the  ADMFs,  which  provide  personnel  inventory  “snapshots”  rather  than  just  transaction 
records. 
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Coast  Guard  personnel  are  included  starting  in  1989. 

Enclosure  3  in  DoDI  1336.05  Automated  Extract  of  Active  Duty  Military  Personnel  Records  (U.S.  Department  of 
Defense,  2015b)  and  DoDI  1120.11  Programming  and  Accounting  for  Active  Component  (AC)  Military’  Manpower 
(U.S.  Department  of  Defense,  2015a)  list  the  information  recorded  on  the  active  duty  military  and  active  duty  Coast 
Guard  personnel  who  are  included  in  the  data  file. 
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Coast  Guard  personnel  are  included  starting  in  1989. 
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See  DAWIA  (Title  10  U.S.C.  Ch.  87)  and  DoDI  5000.55  Reporting  Management  Information  on  DoD  Military 
and  Civilian  Acquisition  Personnel  and  Positions  (U.S.  Department  of  Defense,  1991).  DoDI  5000.55  (U.S. 
Department  of  Defense,  1991)  outlines  the  specific  reporting  requirements  for  the  DAW. 
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This  study  uses  the  annual  end-of-FY  DAWIA  records  and  has  been  granted  access  to 
many  of  these  variables,  including — among  other  things — information  on  the  acquisition 
certification  level  each  individual  has  achieved  and  the  defense  acquisition  organization 
and  career  field  each  position  falls  under.  This  research  uses  DAWIA  data  files  starting  in 
FY  1992. 

All  of  these  data  files  use  the  same  personal  identifier  to  identify  specific  individuals.  This 
shared  personal  identifier  among  the  data  files  allows  the  files  to  be  “linked”  so  that  individuals 
can  be  followed  “across  files”  and  “over  time”  (Gates  et  ah,  2008,  p.  6;  Gates  et  ah,  2013,  p.  3; 
Guo,  Hall-Partyka  and  Gates,  2014,  p.  2). 


Defining  a  Cohort 

This  study  defines  a  civilian  DAW  cohort  as  the  group  of  individuals  that  enters  the  civilian 
DAW  in  a  given  FY.  A  merged  dataset  consisting  of  annual  (end-of-FY)  DoD  civilian  personnel 
inventory  files  and  DAWIA  files  is  used  to  detennine  who  enters  the  civilian  DAW  and  in  which 
FY  he  or  she  enters.  The  DoD  civilian  inventory  files  show  which  individuals  are  a  part  of  the 
DoD  civilian  workforce  at  the  end  of  each  FY,  and  the  DAWIA  data  indicate  which  individuals 
are  in  acquisition  positions.  This  analysis  defines  and  examines  cohorts  from  FY  2000  to  FY 
2014. 

An  individual  is  included  in  a  FY  cohort  under  two  circumstances:  (1)  if  he  or  she  is  a 
“transfer”  or  (2)  if  he  or  she  is  a  “new  hire.”  These  two  terms  and  the  conditions  that  define  them 

32 

are  described  below. 

•  “transfers”33 

An  individual  is  considered  a  “transfer”  into  the  civilian  DAW  in  year  t  if  he  or  she  is 
(1)  in  the  larger  DoD  civilian  workforce  in  year  t-1  (in  a  non-acquisition  position)  and  (2) 
is  in  an  acquisition  position  in  the  DoD  civilian  workforce  in  year  t.  This  individual  is 
counted  as  a  civilian  DAW  entrant  in  year  t  (i.e.,  he  or  she  is  included  in  that  FY’s 
cohort). 

Like  Gates  et  al.  (2013),  this  analysis  makes  a  distinction  between  professionals  who 
substantively  transfer  into  the  civilian  DAW  and  those  who  administratively  transfer 
(Gates  et  al.,  2013,  pp.  8-12).  A  “substantive”  transfer  is  when  an  individual  actually 
changes  position  or  job  when  he  or  she  moves  into  the  civilian  DAW  (Gates  et  al.,  2013, 
pp.  8-12).  On  the  other  hand,  an  “administrative”  transfer  is  when  a  DoD  civilian  remains 
in  the  same  position  from  one  year  to  the  next  but  that  position  changes  from  a  non¬ 
acquisition  position  to  an  acquisition  position  (Gates  et  al.,  2013,  pp.  8-12).  In  coding 
terms,  a  substantive  transfer  is  flagged  when  an  individual  transfers  to  an  acquisition 
position  and  this  switch  is  accompanied  by  a  change  in  agency/service,  bureau,  functional 
occupational  group,  or  occupational  series.  A  substantive  transfer  is  also  flagged  if  an 
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These  terms  and  their  definitions  stem  from  Gates  et  al.  (2013)  (pp.  7-12). 

33 

The  below  methodology  for  counting  transfers  is  based  on  Gates  et  al.  (2013) — see  pages  8-12.  Gates  et  al.  (2013) 
mainly  use  the  term  “switches”  (pp.  8-12),  although  on  a  few  occasions,  they  also  use  the  term  “transfers”  (pp.  xiii, 
xv,  35).  This  report  mainly  uses  “transfers.” 
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individual  who  transfers  into  the  civilian  DAW  comes  from  or  goes  into  a  Senior 
Executive  Service  (SES)  position  or  if  that  individual  changes  pay  grade  while  remaining 
in  the  same  pay  plan.  In  all  other  cases,  a  transfer  to  the  civilian  DAW  is  recorded  as 
administrative  (Gates  et  al.,  2013). 34  Only  individuals  who  substantively  transfer  into  the 
civilian  DAW  are  counted  as  entrants  (i.e.,  members  of  a  civilian  DAW  cohort)  for  this 
analysis.  Administrative  transfers  are  not  counted.  This  distinction  is  made  because  those 
employees  who  substantively  transfer  into  the  civilian  DAW  actively  choose  to  join  an 
acquisition-related  position,  whereas  individuals  who  are  administrative  transfers  are 
simply  in  positions  that  are  re-coded  as  acquisition  positions — their  responsibilities  do 
not  change  and  they  have  no  choice  in  the  matter.  Recruiting  and  hiring  policies  can 
influence  substantive  transfers  but  not  individuals  who  are  administratively  switched. 

•  “new  hires”  (Gates  et  al.,  2013,  pp.  xii,  7-8) 

An  individual  is  considered  a  “new  hire”  in  year  t  if  he  or  she  was  not  in  the  DoD 
civilian  workforce  in  year  t-1  but  is  in  the  DoD  civilian  workforce  and  in  an  acquisition 
position  in  year  t  (Gates  et  al.,  2013,  pp.  xii,  7-8).  This  individual  is  counted  as  a  civilian 
DAW  entrant  in  year  t  (i.e.,  he  or  she  is  included  in  that  FY’s  cohort). 

There  are  certain  cases  where  individuals  are  recorded  as  joining  the  DoD  civilian 
workforce  in  a  non-acquisition  positions  in  year  t  and  then  administratively  transferring 
into  an  acquisition  position  the  following  year  (year  t+1).  Many  of  these  cases  may  be 
due  to  a  lag  in  reporting  and  recording  DAWIA  data.  It  is  possible  that  these 
professionals  are  actually  hired  into  a  civilian  DAW  position  in  year  t  but  are  first 
recorded  as  in  an  acquisition  position  the  following  year  (year  t+7).35  Therefore,  in  an 
effort  to  accurately  describe  civilian  DAW  cohorts,  these  individuals  are  included  in  the 
cohort  of  the  FY  in  which  they  were  brought  into  the  DoD  civilian  workforce  (year  t). 
Hereafter,  these  instances  are  referred  to  as  “lag  instances.” 

Figures  6.1  and  6.2,  below,  illustrate  the  civilian  DAW  cohorts  that  this  methodology 

36 

produces  for  FYs  2000-2014.  Figure  6.1  shows  the  overall  cohort  size,  and  Figure  6.2 
illustrates  the  number  of  lag  instances  included  in  each  cohort.  There  are  several  hundred  lag 
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See  Gates  et  al.  (2013)  pp.  8-12  for  a  more  detailed  explanation  on  defining  substantive  and  administrative 
transfers. 

35 

It  is  impossible  to  be  sure  that  this  assertion  holds  in  each  instance.  However,  Gates  et  al.  (2013)  find  other 
“discrepancies”  between  the  DAWIA  files  and  the  active  duty  and  DoD  civilian  files  (p.  14),  making  this  a 
reasonable  assertion.  Specifically,  Gates  et  al.  (2013)  find  that 

some  people  appear  as  military  in  the  DAWIA  file  but  match  only  to  the  end-of-year  civilian 
inventory  file  (or  vice  versa).  Our  review  of  the  cases  indicates  that  the  DAWIA  file  is  not  being 
updated  immediately  to  reflect  changes  in  status  (i.e.,  a  person  separates  from  the  military  and  is 
hired  as  a  civilian)  (p.  14). 

Gates  et  al.  (2013)  hypothesize  that  some  of  these  discrepancies  may  be  due  partially  to  the  fact  that  the  DAWIA 
data  follow  a  different  data-reporting  process  than  do  the  ADMFs  and  the  DoD  civilian  personnel  files.  Specifically, 
Gates  et  al.  (2013)  write  “DoD  components  report  [DAWIA]  data  on  a  quarterly  basis,  while  the  civilian  and 
military  personnel  files  are  updated  on  a  just-in-time  basis,  when  a  transaction  occurs”  (p.  14). 

36 

Individuals  can  be  in  more  than  one  cohort  if  they  enter  the  civilian  DAW  on  more  than  one  occasion  based  on 
the  methodology  used  in  this  study  (which  is  outlined  in  Chapter  6).  In  total,  the  cohorts  consist  of  149,839  unique 
individuals;  7,439  (around  five  percent)  of  them  are  in  more  than  one  cohort.  The  sum  of  the  non-unique 
observations  for  the  FY  2000-2014  cohorts  equals  157,588. 
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instances  included  in  most  FY  cohorts,  with  over  750  in  the  FY  2001  and  FY  2009  cohorts. 
Based  on  the  data,  one  cannot  determine  definitively  whether  each  particular  lag  instance  should 
be  included  in  a  particular  civilian  DAW  cohort;  their  inclusion  simply  represents  a  “best  guess” 
at  interpreting  the  data.  In  any  case,  the  inclusion  of  the  lag  instances  is  not  substantially 
consequential:  Lag  instances  never  make  up  more  than  1 1  percent  of  any  particular  civilian 
DAW  cohort,  and  most  cohorts  consist  of  no  more  than  five  percent  of  lag  instances.  Moreover, 
a  sensitivity  analysis,  presented  in  Appendix  H,  finds  that  the  overall  conclusions  from  Parts  I 
and  II  of  the  retention  analysis  (presented  in  Chapters  8  and  9,  and  in  Appendix  F)  are  not 
sensitive  to  the  inclusion  of  the  lag  instances  in  the  civilian  DAW  cohorts. 

Figure  6.1.  Civilian  DAW  Cohort  Sizes  for  FY  2000-2014  Cohorts 
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Figure  6.2.  Number  of  Lag  Instances  Included  in  Each  Civilian  DAW  Cohort,  FYs  2000-2014 


Characterizing  Past  Work  Experience 

Once  all  of  the  civilian  DAW  cohorts  (from  FY  2000  to  FY  2014)  have  been  defined,  the 
next  step  is  building  a  work  history  for  each  individual  entrant.  This  analysis  uses  a  merged 
dataset  consisting  of  the  end-of-FY  active  duty  files,  the  end-of-FY  DoD  civilian  personnel 
inventory  files,  and  the  end-of-FY  DAWIA  data  to  look  back  at  each  civilian  DAW  entrant’s 

'XI 

past  work  experience  within  the  DoD.  Based  on  the  data,  an  individual  is  categorized  into  one 
of  three  conditions  in  each  FY,  beginning  with  FY  1980.  In  each  FY,  an  individual  is  in  (1)  the 
DoD  civilian  workforce;  (2)  the  active-duty  military;  or  (3)  a  non-DoD  position  (i.e.,  Other). 
Starting  in  FY  1992  (the  first  year  for  which  data  on  the  DAW  are  available),  conditions  1  and  2 
are  divided  into  two  mutually  exclusive  sub-groups  to  capture  DAW  experience:  An  individual 

TO 

who  is  in  the  DoD  civilian  workforce  is  either  in  the  (la)  civilian  DAW  or  in  (lb)  another  DoD 
civilian  position.  An  individual  in  the  active-duty  military  is  either  in  the  (2a)  military  DAW  or 
in  (2b)  another  active-duty  military  position.  Figures  6.3  and  6.4,  below,  illustrate  these 
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Starting  in  FY  1989,  the  active  duty  data  that  this  study  uses  also  include  active  component  Coast  Guard 
personnel.  So  for  this  study,  active  duty  military  experience  includes  experience  in  the  active  component  of  the 
Coast  Guard  starting  in  FY  1989. 
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This  is  also  adjusted  for  DAWIA-data  lag  instances,  in  parallel  with  the  “lag”  adjustment  made  when  defining 
civilian  DAW  cohorts.  Therefore,  an  individual  is  marked  as  in  the  civilian  DAW  in  year  /  under  one  of  two 
conditions: 

(1)  If  he  or  she  appears  in  the  DoD  civilian  file  and  is  recorded  as  in  the  DAW. 

(2)  If  he  or  she  is  recorded  as  joining  the  DoD  civilian  workforce  in  a  non-acquisition  position  in  year  t  and  then 
administratively  transferring  into  an  acquisition  position  the  following  year  (year  t+1).  As  explained  above, 
many  of  these  cases  may  be  due  to  a  lag  in  reporting  and  recording  DAWIA  data.  It  is  possible  that  these 
professionals  are  actually  hired  into  a  civilian  DAW  position  in  year  t  but  are  first  recorded  as  in  an  acquisition 
position  the  following  year  (year  t+1).  These  individuals  are  marked  as  being  in  the  civilian  DAW  in  year  t. 
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conditions.  Figure  6.3  shows  the  three  conditions  that  exist  from  FY  1980  to  FY  1991,  and 
Figure  6.4  illustrates  how  conditions  1  and  2  are  divided  into  their  mutually  exclusive  sub-groups 
starting  inFY  1992. 

This  categorization  scheme  provides  flexibility  to  analyze  the  workforce  and  prior  workforce 
experience  in  different  ways.  One  can  combine  categories  la  and  lb  to  analyze  experience  in  the 
DoD  civilian  workforce  and  categories  2a  and  2b  to  analyze  active  duty  experience.  One  can  also 
combine  categories  la  and  2a  to  examine  all  individuals  with  experience  in  the  DAW  or  combine 
categories  la,  lb,  2a,  and  2b  to  analyze  all  individuals  with  DoD  experience. 

Figure  6.3.  Framework  for  Characterizing  Past  Work  Experience,  FY1980-FY1991 
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Figure  6.4.  Framework  for  Characterizing  Past  Work  Experience,  FY1992-FY2014 
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Adjustments  for  Double  Counting 

Some  individuals  are  recorded  in  two  different  defense  workforce  files  at  the  same  time  (i.e., 
in  the  same  end-of-FY  personnel  inventory  snapshots).  Specifically,  certain  individuals  appeared 
in  both  the  ADMF  and  DoD  civilian  inventory  files  in  the  same  annual  data  cuts.  Although  this  is 
not  a  common  occurrence — the  number  of  individuals  who  appear  in  these  two  files  at  the  same 
time  on  one  or  more  occasions  never  exceeds  40  for  any  of  the  civilian  DAW  cohorts  analyzed 
(which  include  thousands  of  observations) — and  it  is  impossible  to  say  exactly  what  is  causing 
these  “double  counts”  to  occur,  a  decision  had  to  be  made  about  how  to  count  the  work 
experience  for  those  individuals  for  those  years. 

After  examining  the  data,  one  distinct  pattern  became  clear.  It  is  not  uncommon  for  an  active 
duty  service  member  to  join  the  DoD  civilian  workforce  directly  upon  his  or  her  separation  from 
the  active  duty  force.  This  is  seen  in  the  data  as  an  individual  appearing  in  the  ADMF  one  year 
and  then  appearing  in  the  DoD  civilian  inventory  file  the  next  year.  In  some  cases,  however,  it 
seems  that  this  transition  has  some  overlap — creating  this  “double-count”  issue.  In  particular, 
some  individuals  appear  in  the  ADMF  one  year,  then  in  both  the  ADMF  and  DoD  civilian  file 
the  next  year,  and  then  only  in  the  DoD  civilian  file  the  following  year.  This  overlap  could  (1) 
stem  from  a  lag  in  recording  an  individual’s  separation  from  the  active  duty  military  or  (2) 
represent  an  individual  who  is  on  terminal  leave  from  the  active  duty  military  and  has  already 
joined  the  DoD  civilian  workforce.  These  “double-count”  instances  are  adjusted  accordingly 
under  this  assumption.  Specifically,  for  the  “double-count”  cases  that  match  this  pattern, 
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individuals  are  marked  as  only  in  the  DoD  civilian  workforce  (and  not  in  the  active-duty 
military)  during  that  “double-count”  year.39 

For  the  “double-count”  cases  that  do  not  follow  the  pattern  described  above,  individuals  are 
marked  as  only  in  the  active-duty  military  (and  not  in  the  DoD  civilian  workforce).  It  is 
impossible  to  know  what  exactly  is  causing  these  particular  “double-count”  instances  to  occur. 
However,  this  adjustment  seemed  logical  because  one  reasonable  hypothesis  for  some  of  these 
“double-count”  instances  is  as  follows:  Perhaps  some  of  these  cases  stem  from  DoD  civilian 
employees  being  called  to  serve  on  active  duty  (and  being  recorded  in  the  ADMF,  per  DoDI 
1 120. 1 1  (U.S.  Department  of  Defense,  2015a)),  while  they — at  the  same  time — maintain  their 
DoD  civilian  positions  in  the  personnel  files. 

DAWIA  data  are  used  post  adjustment  to  determine  if  individuals  should  be  marked  as  in  the 
DAW  for  each  “double-count”  case.  If  the  adjustments  described  above  mark  an  individual  as 
only  in  the  DoD  civilian  workforce  (and  not  in  the  active-duty  military)  for  a  particular  “double¬ 
count”  instance,  he  or  she  is  determined  to  be  in  the  civilian  DAW  at  that  time  under  two 
circumstances:  (1)  if  the  DAWIA  data  indicate  that  individual  to  be  in  the  DAW  that  year  and 
show  that  individual  to  be  in  the  civilian  DAW  at  that  time  or  (2)  if  that  individual  is  marked  as 
in  the  civilian  DAW  for  that  year  because  of  the  DAWIA  lag  adjustment  described  above. 
Otherwise,  the  individual  is  simply  marked  as  in  the  wider  (non-DAW)  DoD  civilian  workforce 
for  that  particular  “double-count”  instance. 

If  the  adjustments  mark  an  individual  as  only  in  the  active-duty  military  (and  not  in  the  DoD 
civilian  workforce)  for  a  particular  “double-count”  instance,  he  or  she  is  determined  to  be  in  the 
military  DAW  at  that  particular  time  if  the  DAWIA  data  indicate  that  individual  to  be  in  the 
DAW  that  year  and  show  that  individual  to  be  in  the  military  DAW  at  that  time.  Otherwise,  the 
individual  is  simply  marked  as  in  the  wider  (non-DAW)  active-duty  military  for  that  particular 
“double-count”  instance. 

The  following  fictional  example  illustrates  this  adjustment  process  in  full.  Assume  that  Jane, 
a  fictional  individual,  is  a  member  of  the  FY  2010  civilian  DAW  cohort.  When  characterizing 
Jane’s  prior  work  experience,  the  data  show  Jane  to  be  in  the  DoD  civilian  workforce  from  1995 
to  2002  (i.e.,  Jane  appears  in  the  end-of-FY  1995-2002  DoD  civilian  personnel  files).  The  data 
also  show  Jane  to  be  in  the  active-duty  military  in  years  1997  and  1998  (i.e.,  Jane  appears  in  the 
end-of-FY  1997-1998  ADMFs).  The  issue,  then,  is  that  Jane  appears  in  the  end-of-FY  1997-1998 
ADMFs  and  DoD  civilian  personnel  files.  Jane’s  “double-count”  instances  in  FYs  1997  and  1998 
do  not  follow  the  overlap  pattern  described  above.  Therefore,  following  the  rule  outlined  above, 
these  “double-count”  instances  for  Jane  are  adjusted  to  show  her  as  only  in  the  active-duty 
military  in  1997  and  1998  (and  not  in  the  DoD  civilian  workforce).  After  this  adjustment  is 
made,  the  DAWIA  data  are  considered.  Jane  does  not  appear  in  the  end-of-FY  1997-1998 

39 

For  the  cases  where  the  “double-count”  year  is  FY  1980,  since  it  is  not  possible  to  look  back  a  year,  an  individual 
is  marked  as  only  in  the  DoD  civilian  workforce  in  FY  1980  if  that  person  is  marked  as  only  in  the  DoD  civilian 
workforce  in  FY  1981.  In  all  other  cases  where  the  “double-count”  year  is  FY  1980,  that  individual  is  recorded  as 
being  only  in  the  active-duty  military  for  FY  1980. 
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DAWIA  data.  For  this  reason,  Jane  is  ultimately  marked  as  being  in  the  wider  (non-DAW) 
active-duty  military  for  FYs  1997  and  1998,  her  “double-count”  years. 

Limitations  to  Methodologies  for  Defining  Civilian  DA  W  Cohorts  and  Characterizing  Past 

Work  Experience 

Using  the  administrative  end-of-FY  personnel  files  to  define  cohorts  and  characterize  past 
work  experience  is  simple  and  effective:  The  annual  files  are  available  for  both  the  civilian  and 
active-duty  military  personnel  going  back  to  end-of-FY  1980.  They  are  intuitive  to  work  with, 
and — as  illustrated  above — they  capture  the  basic  infonnation  needed  for  this  study. 

However,  the  use  of  the  annual  files  does  have  its  limitations.  This  technique  misses  some 
individuals  and  miscategorizes  others.  First,  in  terms  of  defining  who  is  in  each  of  the  civilian 
DAW  cohorts,  the  end-of-FY  data  only  capture  individuals  who  are  in  the  civilian  DAW  at  the 
end  of  each  FY.  Individuals  who  are  not  in  a  civilian  DAW  position  at  the  end  of  a  FY  (from  FY 
2000  to  FY  2014)  are,  therefore,  not  included  in  the  cohort  for  that  FY.  In  particular,  an 
individual  could  join  the  civilian  DAW  in  March  of  one  year  and  then  leave  the  workforce  before 
the  end  of  that  FY.  In  this  case,  this  particular  individual  would  not  be  recorded  as  a  member  of 
that  FY’s  cohort,  as  he  or  she  was  never  recorded  in  an  end-of-FY  inventory  file.  Additionally,  it 
is  important  to  note  that  the  data  and  methodology  used  include  seasonal  employees  in  the 
civilian  DAW  cohorts.  However,  the  number  of  instances  where  this  happens  is  quite  small: 
There  are  only  30  total  instances  of  a  seasonal  employee  being  included  in  the  FY  2000-2014 
civilian  DAW  cohorts,  with  no  single  cohort  having  more  than  eight,  and  there  are  never  fewer 
than  4,000  individuals  in  a  cohort. 

The  use  of  annual  end-of-FY  inventory  files  to  characterize  work  experience  means  that 
individuals  who  are  in  a  particular  DoD  workforce  on  September  30th  of  each  FY  are  counted  has 
having  a  year  of  experience  in  that  workforce.  This  method  gives  certain  individuals,  particularly 
those  who  leave  the  workforce  not  too  long  before  the  end  of  the  FY,  less  experience  in  that 
workforce  than  they  really  had.  On  the  other  hand,  this  method  gives  other  individuals, 
particularly  those  who  enter  a  workforce  right  before  the  end  of  the  FY,  more  experience  in  that 
workforce  than  they  really  had.  This  limitation  is  even  more  salient  when  accounting  for  past 
work  experience  of  individuals  with  a  background  as  a  seasonal  DoD  civilian  employees: 
Seasonal  DoD  civilians  likely  enter  and  leave  the  DoD  civilian  workforce  more  frequently, 
remaining  in  the  workforce  for  shorter  periods  of  time,  thereby  exacerbating  the  over-  and 
undercounting  of  past  DoD  Civilian  experience.40  Additionally,  it  is  also  important  to  note  that 
the  data  and  methodology  used  do  not  distinguish  between  full-time  and  not-full-time  experience 
in  the  DoD  civilian  workforce.41 


40 

The  end-of-FY  1980  and  end-of-FY  1981  fdes  report  no  seasonal  employees.  It  is  not  known  if  the  1980  and 
1981  fdes  contain  no  seasonal  employees  or  if  seasonal  employees  in  these  fdes  were  not  properly  recorded. 

41 

The  DoD  civilian  personnel  data  include  all  types  of  workers,  including  “Full-time,”  “Part-time,”  and 
“Intermittent”  employees  ("DoD  APF  Civ  Personnel  Edit  Unpacked  File  (200803  and  After,  Data  Dictionary)," 
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This  methodology — using  administrative  end-of-FY  inventory  files  to  record  DoD 
experience — does  not  perfectly  illustrate  the  past  work  experience  of  civilian  DAW  entrants:  It 
undercounts  experience  for  some  individuals  and  over  counts  experience  for  others.  However, 
given  that  this  analysis  is  at  an  aggregate  cohort  level  (and  considering  that  there  are  thousands 
of  individuals  in  each  cohort),  these  over-  and  undercounts  may  even  out  in  the  end. 

Additionally,  this  methodology  should  not  significantly  skew  descriptions  of  whether  or  not  a 
professional  has  particular  work  experiences:  Only  individuals  who  have  been  in  a  particular 
workforce  but  never  appear  in  an  end-of-FY  file  for  that  workforce  would  be  falsely  recorded  as 
never  having  been  in  that  workforce.  Theoretically,  then,  this  “false  negative”  could  only  occur 
in  instances  where  individuals  are  in  a  workforce  for  less  than  one  year  at  one  time,  and  it  is 
unlikely  that  a  single  year  of  experience  (or  less)  in  a  particular  workforce  would  be  significantly 
influential.  In  the  end,  this  technique  provides  a  good  approximation  of  the  past  DoD  work 
experience  of  civilian  DAW  entrants. 

Past  Work  Experience  Information  for  FY  Cohorts  2000-2014 

Once  each  civilian  DAW  entrant  is  categorized  into  one  of  the  workforces  in  each  FY  (from 
FY  1980  to  the  FY  prior  to  him  or  her  joining  the  civilian  DAW)  based  on  the  methodology 
described  above,  indicator  variables  are  created  that  flag  what  prior  work  experiences  each 
individual  has  had.  The  prior-DoD-experience  indicator  variables  are  not  mutually  exclusive:  An 
entrant  could  have  prior  experience  in  the  active-duty  military  non-DAW  (AD),  in  the  wider 
non-DAW  DoD  civilian  workforce  (DoD  Civ),  in  the  military  DAW  (Mil  DAW),  and/or  in  the 
civilian  DAW  (Civ  DAW).  Some  individuals,  however,  join  the  civilian  DAW  with  no  prior 
experience  in  the  DoD  (No  DoD).  The  statistics  for  these  prior-DoD-experience  indicator 
variables  are  shown  in  table  6.1,  below.  These  are  the  data  used  in  the  descriptive  analysis 
discussed  in  Chapter  7. 

Table  6.1.  Prior-DoD-Experience  Indicator  Variable  Means,  FY  Cohorts  2000-2014 


2000 

2001 

2002 

2003 

2004 

2005 

FY  Cohorts 

2006  2007  2008 

2009 

2010 

2011 

2012 

2013 

2014 

N 

4433 

8487 

10356 

8966 

10717 

10220 

8044 

8680 

12448 

17308 

19073 

12843 

9738 

8019 

8256 

AD 

% 

21.09 

21.2 

24.77 

26.05 

24.62 

29.77 

32.81 

33.74 

33.62 

31.88 

33.46 

37.04 

39.03 

38.85 

38.94 

DoD  Civ 

% 

58.85 

68.71 

60.76 

55.64 

60.04 

57.78 

48.62 

45.81 

42.92 

32.67 

29.59 

39.06 

44.65 

47.71 

45.55 

Mil  DAW 

% 

2.41 

2.25 

2.91 

4.07 

3.48 

4.45 

5.64 

6.79 

5.6 

5.59 

5.94 

6.53 

4.72 

5.51 

6.2 

Civ  DAW 

% 

22.29 

14.96 

12.06 

15.44 

16.54 

13.34 

12.49 

13.78 

14.09 

9.93 

9.43 

12.07 

12.88 

13.12 

14.53 

No  DoD 

% 

28.38 

19.74 

25.44 

27.93 

25.95 

25.19 

30.37 

32.76 

34.94 

44.96 

45.21 

35.39 

30.84 

30.12 

31.87 

2010,  p.  4).  “Intermittent”  employees  are  those  individuals  who  do  not  have  a  regular,  set  “Work  Schedule”  ("DoD 
APF  Civ  Personnel  Edit  Unpacked  File  (200803  and  After,  Data  Dictionary),"  2010,  p.  4). 
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The  Outcome  Variable — Retention 


After  defining  the  civilian  DAW  cohorts  and  characterizing  prior  work  experience,  the 
analysis  adds  the  necessary  tenure  information  and  control  factors  in  order  to  examine  retention 
trends.  These  retention  analyses  focus  on  the  pre -policy  cohorts  (FY  cohorts  2001-2005  and 
2007)  42  and  the  policy  cohorts  (FY  cohorts  2008-2011).  The  aim  of  the  retention  analyses  is  to 
understand  civilian  DAW  retention  trends — how  these  trends  might  differ  across  individuals 
with  different  past  work  experiences  and  between  pre-policy  and  policy  cohorts.  To  this  end, 
retention  analysis  Part  I  also  investigates  retention  in  the  wider  non-DAW  DoD  civilian 
workforce.  This  tangential  investigation  into  retention  in  the  wider  DoD  civilian  workforce  helps 
to  further  clarify  the  relationship  between  prior  work  experience  and  retention  in  the  civilian 
DAW.  The  following  sections  define  retention  in  the  wider  DoD  civilian  workforce  and  in  the 
civilian  DAW,  list  and  explain  the  control  factors  included  in  the  retention  analyses,  and  outline 
the  final  data  file  used  for  the  retention  analyses. 

Defining  Retention  in  the  Wider  DoD  Civilian  Workforce 

An  individual  is  marked  as  a  loss  from  the  DoD  civilian  workforce  at  the  end  of  FY  t  if  he  or 
she  is  in  the  DoD  civilian  workforce  at  the  end  FY  t  but  not  in  the  DoD  civilian  workforce  at  the 
end  of  FY  t+1.  Figure  6.5  provides  an  illustration. 

Figure  6.5.  This  Individual  is  Marked  as  a  DoD  Civilian  Loss  at  the  end  of  FY  t 


FY  t  FY  t+1 


DAW  DAW 

DoD  Civilian  WF  DoD  Civilian  WF 


Each  individual’s  tenure  in  the  DoD  civilian  workforce  is  calculated  by  subtracting  his  or  her 
civilian  DAW  cohort  FY  for  that  particular  instance  (since  some  individuals  are  in  multiple 
civilian  DAW  cohorts)  from  the  FY  at  the  end  of  which  he  or  she  is  recorded  as  a  loss  (in  Figure 
6.5,  this  would  be  FY  t).  For  example,  an  individual  who  is  in  the  FY  2003  civilian  DAW  cohort 
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The  FY  2006  cohort  is  dropped  from  the  retention  analyses  because  a  large  number  of  observations  in  this  cohort 
do  not  have  information  for  the  race  variable,  as  described  below. 
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and  is  recorded  as  a  loss  at  the  end  of  FY  2010  has  a  tenure  in  the  DoD  civilian  workforce  of 
seven  years. 


Defining  Retention  in  the  Civilian  DA  W 

Defining  retention  in  the  civilian  DAW  is  more  complicated  because  there  are  three  different 
types  of  civilian  DAW  losses  that  individuals  can  experience  (Gates  et  ah,  20 13). 43  An  individual 
can  separate  from  the  civilian  DAW,  which  involves  leaving  the  DoD  civilian  workforce 
altogether.  Or  an  individual  can  transfer — either  substantively  or  administratively — out  of  the 
civilian  DAW  (Gates  et  ah,  2013).  These  three  loss  types  are  described  in  more  detail  below  and 
illustrated  in  Figure  6.6. 

Figure  6.6.  There  are  Three  Different  Types  of  Losses  from  the  Civilian  DAW 
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•  “separations”  from  the  civilian  DAW  (Gates  et  ah,  2013,  pp.  7-8) 

An  individual  is  marked  as  separating  from  the  civilian  DAW  at  the  end  of  FY  t  when 
he  or  she  is  in  the  civilian  DAW  one  year  (in  FY  t)  and  not  in  the  DoD  civilian  workforce 
at  all  in  the  following  year  (FY  t+1)  (Gates  et  ah,  2013,  p.  8). 

•  “transfers”  out  of  the  civilian  DAW44 


43  The  definitions  of  these  three  loss  types  come  from  Gates  et  al.  (2013)  (pp.  7-12). 

44 

Gates  et  al.  (2013)  mainly  use  the  term  “switches”  (pp.  8-12)  instead  of  “transfers.”  However,  on  a  few  occasions, 
they  also  use  the  term  “transfers”  (pp.  xiii,  xv).  This  report  mainly  uses  “transfers.” 
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An  individual  is  considered  a  transfer  out  of  the  civilian  DAW  at  the  end  of  FY  t  if  he 
or  she  (1)  is  in  a  civilian  defense  acquisition  position  in  FY  t  and  (2)  is  in  a  wider  (non¬ 
acquisition)  DoD  civilian  position  in  FY  t+1  (Gates  et  ah,  2013). 

Like  Gates  et  al.  (2013),  this  analysis  makes  a  distinction  between  professionals  who 
substantively  transfer  out  of  the  civilian  DAW  and  those  who  administratively  transfer 
(Gates  et  ah,  2013,  pp.  8-12).  A  “substantive”  transfer  is  when  an  individual  actually 
changes  position  or  jobs  when  he  or  she  moves  out  of  the  civilian  acquisition  workforce 
(Gates  et  al.,  2013,  pp.  8-12).  On  the  other  hand,  an  “administrative”  transfer  is  when  a 
DoD  civilian  remains  in  the  same  position  from  one  year  to  the  next  but  that  position 
changes  from  an  acquisition  position  to  an  non-acquisition  position  (Gates  et  al.,  2013, 
pp.  8-12).  In  coding  tenns,  a  substantive  transfer  is  flagged  when  a  transfer  out  of  the 
civilian  DAW  is  accompanied  by  a  change  in  agency/service,  bureau,  functional 
occupational  group,  or  occupational  series.  A  substantive  transfer  is  also  flagged  if  an 
individual  transferring  out  of  the  civilian  DAW  is  coming  from  or  going  into  an  SES 
position  or  if  that  individual  changes  pay  grade  while  remaining  in  the  same  pay  plan.  In 
all  other  cases,  a  transfer  to  the  civilian  DAW  is  recorded  as  administrative  (Gates  et  al., 
2013). 

The  distinction  between  substantive  and  administrative  transfers  is  made  because 
substantive  transfers  involve  actual  personnel  movement,  whereas  administrative 
transfers  are  simply  bureaucratic  re-categorizations. 

Each  individual’s  tenure  in  the  civilian  DAW  workforce  is  calculated  by  subtracting  his  or 
her  civilian  DAW  cohort  FY  for  that  particular  instance  (since  some  individuals  are  in  multiple 
civilian  DAW  cohorts)  from  the  FY  at  the  end  of  which  he  or  she  is  recorded  as  a  loss  from  the 
civilian  DAW  (in  Figure  6.6,  this  would  be  FY  t ).  For  example,  a  professional  who  is  in  the  FY 
2004  civilian  DAW  cohort  and  is  recorded  as  a  loss  from  the  civilian  DAW  at  the  end  of  FY 
2009  has  a  civilian  DAW  tenure  of  five  years. 


Control  and  Balance  Variables 

In  order  to  understand  the  relationship  between  prior  work  experience  and  workforce 
retention,  the  analyses  must  consider  and  control  for  confounding  factors,  i.e.,  observable 
characteristics  that  relate  to  both  prior  work  experience  and  retention.  For  example,  certain 
characteristics,  such  as  age  and  education,  vary  systematically  with  prior  work  experience,  and 
these  factors  also  influence  retention.  The  following  analyses  control  for  these  factors  in  order  to 
uncover  the  true  relationship  between  prior  work  experience  and  workforce  retention. 

The  DoD  personnel  data  files  used  in  this  study  contain  a  broad  range  of  personnel 
characteristics,  many  of  which  conceivably  affect  retention.  Table  6.2  includes  a  list  of  all  factors 
included  in  this  study  and  a  brief  argument  about  why  each  factor  is  included  as  a  control.  These 
variables  are  called  observable  characteristics. 


In  contrast  to  Guo,  Hall-Partyka  and  Gates  (2014),  this  study  includes  observations  that  experience  administrative 
losses  in  the  analyses. 
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Table  6.2.  Control  and  Balance  Variables 


Variable45  Description46 


Note 


Compensation  “The  amount  of  annualized  basic  One  would  expect  entrants  with  higher  base  pay  to  have  greater 
pay”  (p.  82)  upon  entrance  into  the  retention.  Guo,  Hall-Partyka  and  Gates  (2014)  show  that 
civilian  DAW.  This  does  not  include  compensation  is  positively  related  to  retention, 
bonuses  or  allowances. 


Age 


Race 


The  age,  in  years,  of  the  entrant 
upon  joining  the  civilian  DAW. 


Indicates  whether  the  entrant  is 
“white”  or  “other.” 


Age  upon  entrance  should  influence  workforce  retention,  but  this 
relationship  may  differ  at  different  age  ranges.  For  example,  the 
youngest  entrants  likely  experience  more  churn;  middle-age 
entrants  may  be  more  attached  to  the  workforce;  and  older 
entrants  are  likely  closer  to  retirement.  Guo,  Hall-Partyka  and 
Gates  (2014)  estimate  that  age — at  an  overall  level — has  an 
inverse  relationship  with  retention. 

Although  it  seems  plausible  that  race  could  relate  to  workforce 
retention,  Guo,  Hall-Partyka  and  Gates  (2014)  find  that  race  is  not 
related  to  retention  in  the  civilian  DAW  once  they  control  for  other 
factors. 


Education 


DoD 

Organization 


Gender 


“Education  Level”  (p.  25)  upon 
entrance  to  the  civilian  DAW.  Marks 
individuals  as  having  (1)  less  than  a 
Bachelor’s  degree,  (2)  a  Bachelor’s 
degree,  or  (3)  more  than  a 
Bachelor’s  degree. 


Individuals  with  different  levels  of  education  likely  have  different 
job  opportunities  outside  of  the  DoD  civilian  workforce/civilian 
DAW.  Therefore,  one  would  expect  education  level  to  influence 
how  long  an  individual  stays  in  the  workforce.  Guo,  Hall-Partyka 
and  Gates  (2014)  find  that  entrants  who  join  the  civilian  DAW  with 
a  Bachelor’s  or  Master’s  degree  tend  to  have  higher  retention 
than  those  with  no  higher-education  degree  (i.e.,  less  than  a 
Bachelor’s  degree).  Asch  (2001),  in  an  analyses  on  the  wider 
DoD  civilian  workforce,  also  found  that  entrants  with  a  Bachelor’s 
degree  tend  to  have  greater  retention  in  the  DoD  civilian 
workforce  than  entrants  with  no  Bachelor’s  degree,  but  Asch 
(2001)  found  no  significant  difference  between  the  retention 
trends  of  entrants  with  a  Master’s  degree  upon  entrance  and 
those  with  no  higher-education  degree  (Guo,  Hall-Partyka  and 
Gates,  2014). 


The  DoD  organization  the  entrant 
joins.  The  four  categories  are:  (1) 
Army,  (2)  Navy,  (3)  Air  Force,  (4) 
Other.  The  Marine  Corps  falls  under 
Navy. 


Each  of  the  DoD  organizations  manages  its  civilian  DAW 
differently.  Therefore,  it  is  plausible  that  the  gaining  institution 
could  influence  workforce  retention.  Guo,  Hall-Partyka  and  Gates 
(2014)  find  some  evidence  suggesting  that  entrants  into  the 
Army’s  civilian  DAW  have  lower  retention  compared  to  entrants 
into  the  “other”  DoD  organizations  (i.e.,  the  non-military- 
department  DoD  organizations). 


The  gender  (male  or  female)  of  It  is  possible  that  men  and  women  have  different  retention  trends, 
each  entrant.  However,  Guo,  Hall-Partyka  and  Gates  (2014)  find  no  relationship 

between  retention  and  gender  once  other  factors  are  controlled 
for. 


45 

Although  they  are  not  quoted  and  indicated  with  a  page  number,  some  of  the  variable  names  listed  below  come 
directly  from  the  DoD-civilian-file  data  dictionary  ("DoD  APF  Civ  Personnel  Edit  Unpacked  File  (200803  and 
After,  Data  Dictionary),"  2010). 

46 

The  DoD-civilian-file  data  dictionary  ("DoD  APF  Civ  Personnel  Edit  Unpacked  File  (200803  and  After,  Data 
Dictionary),"  2010)  outlines  the  variables  available  in  the  DoD  civilian  fde  and  includes  a  description  of  these 
variables.  This  study  has  access  to  a  number  of  the  variables  listed  in  the  dictionary.  Phrases  from  the  data  dictionary 
that  are  used  in  the  description  below  are  quoted  and  indicated  with  a  page  number  ("DoD  APF  Civ  Personnel  Edit 
Unpacked  File  (200803  and  After,  Data  Dictionary),"  2010).  Although  some  of  the  particular  values  listed  in  the 
description  below  also  come  directly  from  the  data  dictionary,  they  are  not  quoted  and  indicated  with  a  page  number 
("DoD  APF  Civ  Personnel  Edit  Unpacked  File  (200803  and  After,  Data  Dictionary),"  2010). 

The  acquisition  career  field  variable  comes  from  the  acquisition  position  data  file. 
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Variable45  Description46  Note 

Position  Type  Indicates  which  type  of  position  an  Each  type  of  position  offers  a  different  experience,  which  could 

individual  joins:  (1 )  Administrative,  conceivably  influence  retention.  This  could  be  especially  true 
(2)  Professional,  (3)  Technical,  (4)  when  considering  administrative  exits  from  the  civilian  DAW.  One 
Other.  hypothesis  would  be  that  administrative  employees  are  more 

likely  to  administratively  transfer  out  of  the  civilian  DAW — that 
they  work  more  at  the  margins  of  the  civilian  DAW. 

Supervisor  Indicates  whether  an  entrant  is  in  a  Guo,  Hall-Partyka  and  Gates  (2014)  find  that  entrants  who  join 

“supervisory”  (p.24)/managerial/  the  civilian  DAW  in  a  supervisory  position  tend  to  have  lower 
leadership  position.  retention  than  those  who  join  in  a  non-supervisory  position. 

Acquisition  Indicates  which  acquisition  career  Different  acquisition  career  fields  have  different  requirements  and 

Career  Field  field  an  entrant  joins  upon  entrance,  responsibilities,  leading  to  contrasting  experiences  across  career 

In  cases  where  an  entrant  is  fields  and  the  expectation  that  retention  rates  would  also  vary, 

missing  a  career  field  indicator  in  Guo,  Hall-Partyka  and  Gates  (2014)  find  that  retention  does  vary 

the  cohort  year,  the  career  field  across  acquisition  career  fields, 

from  the  following  year  is  used  (if 
available).  Guo,  Hall-Partyka  and 
Gates  (2014)  use  a  similar 
technique  to  fill  missing  career-field 
information  (see  p.  10).  The 
Industrial  Property  Management, 

Purchasing,  and  Science  and 
Technology  career  fields  are 
marked  as  “Other.” 

Retirement  Plan  Describes  each  entrant’s  The  retirement  plans  differ  in  how  benefits  are  calculated  and 

“retirement  plan”  (p.  15):  (1)  CSRS,  paid,  thereby  changing  the  incentive  structure  for  remaining  in  the 

(2)  FERS,  (3)  Other.  civil-service  workforce  and  influencing  how  long  individuals 

choose  to  remain  in  the  DoD  civilian  workforce  and  in  the  civilian 
DAW. 

Each  geographic  region  is  unique  in  terms  of  the  local  economy 
and  job-opportunity  environment.  One  would  hypothesize  that 
entrants  in  regions  with  strong  economies  and  higher-paying  job 
opportunities  would  tend  to  have  lower  retention  rates  than 
entrants  in  regions  that  are  struggling  economically. 

Career  Level  Indicates  whether  an  entrant  joins  One  would  expect  that  the  level  of  responsibility  that  an  individual 

Upon  Entrance  as  an  (1)  entry-level,  (2)  mid-level,  has  upon  entering  an  organization  would  relate  to  how  long  that 

or  (3)  senior/executive  employee.  individual  remains  in  the  workforce.  Guo,  Hall-Partyka  and  Gates 

Some  entrants  are  marked  as  (2014)  find  that  mid-level  entrants  tend  to  have  higher  retention 

“uncategorized.”  See  Appendix  J  for  than  entrants  who  join  as  entry-level  employees, 
the  definitions  of  these  career 
levels. 

Cohort  Year  Marks  which  FY  cohort  each  entrant  The  FY  in  which  the  hire  occurred  could  potentially  influence 
is  in.  retention  in  a  couple  of  ways.  First,  the  economic  environment 

has  changed  over  the  years — individuals  in  the  civilian  DAW 
would  likely  be  less  willing  to  leave  the  DoD  civilian  workforce  for 
the  private  sector  during  FY  2008  and  for  several  years  thereafter 
because  of  the  struggling  economy.  Second,  civil-service 
workforce  management  has  also  changed,  and  this  could  also 
influence  separations  and  transfers  out.  Guo,  Hall-Partyka  and 
Gates  (2014)  find  that  retention  trends  do  vary  across  cohort 
years. 


Region  The  geographic  region  (p.  56)  in 

which  the  entrant  is  brought  into  the 
civilian  DAW.  There  are  ten  regions. 
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Variable45  Description46 


Note 


Years  to 

Retirement 

Eligibility 


Pay  Plan 


Handicap 


Describes  how  many  years  an 
entrant  has  until  he  or  she  is  eligible 
for  retirement  benefits.  Categories 
are:  (1)  twenty-plus  years,  (2)  ten  to 
nineteen  years,  (3)  five  to  nine 
years,  (4)  fewer  than  5  years,  (5) 
already  eligible  for  retirement. 


Indicates  which  “pay  plan”  (p.  20) 
an  individual  has:  (1)  Broad  GS,  (2) 
NSPS,  (3)  Acquisition 
Demonstration,  or  (4)  Other. 


One  would  expect  entrants  who  are  already  eligible  for  retirement 
to  have  lower  retention  rates  than  other  entrants.  On  the  other 
extreme,  the  incentives  are  such  that  entrants  who  have  fewer 
than  five  years  until  they  are  fully  eligible  for  retirement  would 
have  very  high  retention  rates  in  those  five  years.  This  factor 
likely  plays  a  major  role  in  workforce  retention.  Although  “Years  to 
Retirement  Eligibility”  is  likely  correlated  closely  with  age,  this 
variable  captures  the  unique,  dichotomous  effect  of  one  being  or 
not  being  eligible  for  retirement. 

Pay  plans  define  the  relationship  between  tenure,  performance, 
and  pay.  Each  of  the  pay  plans  outlines  a  different  incentive 
structure.  These  incentive  structures  undoubtedly  play  a  role  in 
how  long  an  individual  may  stay  in  the  workforce. 


Indicates  which  individuals  have  a  Although  the  vast  majority  of  the  entrants  included  in  this  study  do 
handicap.  not  have  a  handicap,  it  is  important  to  include  this  characteristic 

as  a  control  factor  as  individuals  with  a  handicap  may  be  (1)  more 
or  less  likely  to  have  certain  past  work  experiences  and  (2)  may 
tend  to  have  certain  retention  trends.  Guo,  Hall-Partyka  and 
Gates  (2014)  show  that  there  is  some  evidence  suggesting  that 
handicapped  individuals  tend  to  have  lower  retention  rates  than 
non-handicapped  individuals. 


Retention  Analysis  Data  File 

The  data  file  used  for  the  retention  analyses  combines  all  of  the  factors  described  above: 
prior  work  experience  information,  retention  data,  and  the  other  observable  characteristics.  The 
file  includes  the  pre-policy  cohorts  (FY  cohorts  2001-2005  &  2007)47  and  the  policy  cohorts  (FY 
2008-2011).  The  entire  FY  2006  cohort,  and  a  number  of  observations  in  each  of  the  other 
cohorts  are  dropped  due  to  missing  values  for  one  or  more  of  the  covariates  listed  in  Table  6.2. 

The  analysis  drops  any  individual  that  (1)  has  an  abnormally  low  compensation  amount;  (2) 
does  not  have  a  years-until-retirement  descriptor;  (3)  does  not  have  a  valid  position  type;  (4)  has 
missing  education  information;  (5)  does  not  have  a  race  indicator;  (6)  does  not  have  a  gender 
indicator;  (7)  is  not  a  full-time  employee;  or  (8)  is  missing  a  handicap  indicator.  These 
specifications  drop  a  number  of  observations  from  the  final  analysis  dataset  used  for  the  retention 
analyses,  but — with  the  exception  of  the  FY  2006  cohort — the  vast  majority  of  observations 
remain.  Table  6.3  shows  the  number  of  observations  in  each  cohort  before  and  after  these 
adjustments.  A  significant  number  of  individuals  in  the  FY  2006  cohort  are  missing  a  race 
indicator,  which  is  why  the  analysis  drops  more  than  36  percent  of  the  FY  2006  entrants. 

Because  of  this  missing  information,  the  retention  analyses  do  not  include  the  FY  2006  cohort. 


47 

The  FY  2000  cohort  is  not  included  in  the  base  retention  analysis  data  file  because  the  FY  2000  cohort  is  much 
smaller  than  the  other  pre-policy  cohorts.  Given  the  number  of  entrants  in  FY  2000  relative  to  the  other  cohorts 
examined,  the  FY  2000  cohort  may  be  a  part  of  the  end  of  the  workforce  drawdowns  of  the  1990s  (see  Chapter  2). 

48 

There  are  some  observations  that  have  an  annualized  basic  compensation  of  $100  or  less.  This  analysis  assumes 
that  these  extremely  low  compensation  levels  are  data-entry  errors,  so  these  observations  are  dropped  from  the 
sample. 
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Among  the  other  cohorts  included,  the  adjustments  do  not  drop  more  than  five  percent  of  the 
original  number  of  observations  in  each  cohort. 


Table  6.3.  Dropped  Data  Due  to  Missing  Information 


FY  Cohort 


2001 

2002 

2003 

2004 

2005 

2006 

2007 

2008 

2009 

2010 

2011 

Total  Entrants 

8487 

10356 

8966 

10717 

10220 

8044 

8680 

12448 

17308 

19073 

12843 

Dropped 

Observations 

(106) 

(133) 

(346) 

(447) 

(254) 

(2948) 

(273) 

(610) 

(607) 

(550) 

(457) 

Entrants  in  Analysis 
Dataset 

8381 

10223 

8620 

10270 

9966 

5096 

8407 

11838 

16701 

18523 

12386 

Percentage  Dropped 

1 .2% 

1 .3% 

3.9% 

4.2% 

2.5% 

36.6% 

3.1% 

4.9% 

3.5% 

2.9% 

3.6% 

The  final  dataset  used  for  the  retention  analyses  is  presented  in  Table  6.4,  below.49  The 
cohort  characteristics  do  not  change  significantly  after  dropping  observations  (see  Appendix  A). 


Table  6.4.  Final  Dataset  for  Retention  Analyses 


FY  Cohort 


2001 

2002 

2003 

2004 

2005 

2007 

2008 

2009 

2010 

2011 

All 

N 

8381 

10223 

8620 

10270 

9966 

8407 

11838 

16701 

18523 

12386 

AGE 

Mean 

42 

40 

40 

40 

39 

39 

39 

38 

40 

40 

COMPENSATION3 

Mean 

51812 

51121 

52102 

54177 

52496 

55189 

56120 

57938 

62902 

63311 

RACE 

Other 

% 

21.94 

23.8 

23.09 

21.56 

23.93 

21.23 

23.66 

20.9 

19.9 

20.19 

White 

% 

78.06 

76.2 

76.91 

78.44 

76.07 

78.77 

76.34 

79.1 

80.1 

79.81 

GENDER 

Male 

% 

61.51 

64.83 

65.29 

66.88 

61.78 

65.15 

66.07 

67.18 

68.04 

68.35 

Female 

% 

38.49 

35.17 

34.71 

33.12 

38.22 

34.85 

33.93 

32.82 

31.96 

31.65 

SUPERVISORY  CODE 

Spvsr/Mngr/Ldr 

% 

9.95 

6.96 

7.65 

8.36 

7.84 

8.29 

6.97 

5.95 

6.4 

8.38 

All  Other  Positions 

% 

90.05 

93.04 

92.35 

91.64 

92.16 

91.71 

93.03 

94.05 

93.6 

91.62 

EDUCATION 

Less  than  BA 

% 

36.82 

31.56 

31.42 

29.44 

31.71 

24.93 

24.41 

23.27 

24.27 

24.5 

BA 

% 

41.53 

47.86 

46.26 

47.16 

45.94 

48.35 

50.63 

49.41 

46.28 

44.49 

More  than  BA 

% 

21.64 

20.58 

22.32 

23.4 

22.36 

26.72 

24.96 

27.32 

29.46 

31 

HANDICAP 

Not  Handicapped 

% 

91.54 

90.82 

91.55 

92.04 

92.43 

91.99 

91.97 

91.59 

92.37 

91.02 

Handicapped 

% 

8.46 

9.18 

8.45 

7.96 

7.57 

8.01 

8.03 

8.41 

7.63 

8.98 
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Table  6.4  does  not  show  the  past  work  experience  information  as  these  data  are  already  described  in  Table 


6.1. 
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FY  Cohort 


2001 

2002 

2003 

2004 

2005 

2007 

2008 

2009 

2010 

2011 

DOD  ORGANIZATION 

Air  Force 

% 

12.95 

13.53 

19.14 

12.99 

17.91 

14.33 

13.45 

19.81 

19.25 

23.27 

Army 

% 

53.12 

35.18 

36.91 

57.47 

44.21 

30.28 

36.92 

30.96 

32.33 

29.78 

Navy 

% 

28.39 

39.16 

34.11 

21.91 

19.88 

36.09 

34.06 

34.91 

31.65 

32.21 

Other 

% 

5.55 

12.14 

9.84 

7.63 

18 

19.29 

15.58 

14.32 

16.77 

14.74 

REGION 

Other  or  Missingb 

% 

3.76 

2.65 

2.47 

2.65 

2.82 

1.86 

2.33 

1.87 

2.21 

2.87 

New  England 

% 

3.63 

4.21 

4.15 

3.36 

3.44 

3.45 

2.85 

3.96 

3.79 

3.21 

Mid  Atlantic 

% 

10.45 

10.17 

9.44 

11.39 

11.17 

12.63 

12.62 

9.04 

7.63 

5.83 

East  North  Central 

% 

10.75 

9.28 

11.64 

12.65 

13.99 

10.8 

11.93 

14.96 

13.54 

13.04 

West  North  Central 

% 

1.16 

1.51 

1.4 

1.92 

1.71 

1.14 

1.28 

1.12 

1.01 

1.34 

South  Atlantic 

% 

32.26 

35.91 

33.16 

34 

32.17 

39.16 

35.01 

37.22 

37.56 

40.37 

East  South  Central 

% 

11.72 

6.8 

7.12 

9.27 

8.6 

8.41 

7.81 

7.08 

11.73 

9.47 

West  South  Central 

% 

7.55 

6.55 

7.82 

7.1 

7.31 

4.47 

6.38 

6.18 

5.3 

7.08 

Mountain 

% 

6.62 

5.98 

7.02 

6.86 

7.2 

5.64 

4.62 

5.54 

5.46 

5.33 

Pacific 

% 

12.1 

16.94 

15.78 

10.8 

11.59 

12.44 

15.17 

13.02 

11.76 

11.47 

POSITION  TYPE 

Administrative 

% 

43.78 

36.66 

40.13 

37.36 

41.85 

37.65 

38.6 

36.99 

43.04 

43.44 

Other 

% 

1.26 

1.52 

1.35 

1.15 

1.1 

1.15 

0.71 

0.96 

0.43 

1.6 

Professional 

% 

49.99 

56.85 

53.11 

55.88 

51.54 

55.57 

54.84 

56.92 

51.99 

50 

Technical 

% 

4.96 

4.97 

5.42 

5.61 

5.51 

5.63 

5.85 

5.13 

4.53 

4.96 

ACQ  CAREER  FIELD 

Business  (CE  or  FM) 

% 

10.89 

6.58 

9.51 

6.16 

8.59 

7.71 

6.87 

7.69 

7.78 

7.6 

Contracting 

% 

13.41 

17.34 

14.07 

13.46 

17.27 

20.29 

21.32 

21.87 

18.78 

17.6 

Engineering 

% 

27.71 

30.63 

27.92 

24.59 

17.57 

22.73 

19.76 

23.25 

20.86 

20.64 

Facilities  Engineering 

% 

0.23 

2.06 

6.22 

13.58 

3.6 

4.4 

7.03 

6.61 

5.44 

6.32 

Information  Tech 

% 

5.49 

9.03 

6.42 

3.65 

4.31 

3.96 

4.18 

4.2 

6.17 

6.64 

Life  Cycle  Logistics 

% 

12.35 

7.85 

9 

10.95 

14.36 

12.94 

14.87 

12.95 

14.37 

14.45 

Prod,  Quality  &  Manuf 

% 

3.65 

3.97 

5.64 

6.42 

7.93 

7.62 

7.47 

6.16 

5.69 

6.04 

Program  Management 

% 

9.58 

8.72 

7.71 

7.8 

7.67 

6.53 

6.68 

7.62 

11.2 

10.97 

Test  and  Eval 

% 

4.12 

5.44 

5.52 

4.4 

4.01 

4.48 

4.38 

3.53 

4.34 

3.52 

Other 

% 

3.78 

4.08 

3.57 

3.27 

2.6 

3.25 

2.24 

2.93 

2.05 

3.23 

Unknown  or  Missing 

% 

8.79 

4.3 

4.41 

5.72 

12.09 

6.08 

5.21 

3.19 

3.32 

2.99 

RETIREMENT  PLAN 

CSRS 

% 

26.56 

17.4 

17.03 

14.85 

10.65 

6.53 

5.64 

3.32 

2.72 

2.85 

FERS 

% 

72.9 

82.1 

82.29 

84.4 

88.42 

92.67 

93.32 

95.71 

96.41 

96.04 

Other 

% 

0.54 

0.5 

0.68 

0.75 

0.93 

0.8 

1.04 

0.98 

0.87 

1.11 

CAREER  LEVEL 

Entry-Level 

% 

21.3 

28.5 

29.07 

29.99 

30.03 

33.96 

35.06 

36.65 

25.02 

21.4 

Sen/Exec 

% 

7.78 

5.91 

5.56 

6.78 

6 

9.74 

9.65 

9.36 

9.84 

11.08 

Mid-Level 

% 

70.34 

64.4 

64.47 

61.97 

63.02 

55.42 

54.51 

53.43 

64.66 

65.83 

Uncategorized 

% 

0.58 

1.18 

0.9 

1.27 

0.94 

0.88 

0.79 

0.56 

0.48 

1.69 
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2001 

2002 

2003 

2004 

FY  Cohort 

2005  2007 

2008 

2009 

2010 

2011 

PAY  PLAN 

Other 

% 

0.51 

0.98 

0.92 

1.24 

0.87 

0.79 

0.62 

1.53 

1.25 

2.13 

Demo 

% 

15.51 

21.5 

19.33 

19.66 

16.54 

13.89 

12.72 

11.05 

11.81 

21.87 

Broad  GS 

% 

83.98 

77.52 

79.76 

79.1 

82.59 

70.43 

58.71 

50.85 

68.58 

75.9 

NSPS 

% 

14.89 

27.95 

36.57 

18.36 

0.1 

YRS  UNTIL  RET  ELIG 

20+  YRs  To  Go 

% 

32.76 

42.08 

42.02 

43.7 

45.06 

48.25 

48.96 

51.92 

47.66 

45.79 

10  to  19  Yrs  to  Go 

% 

36.13 

33.18 

31.87 

30.96 

31.65 

31.09 

30.11 

28.95 

30.59 

31.45 

5-9  Yrs  to  Go 

% 

16.62 

13.31 

13.71 

12.77 

11.63 

10.93 

10.97 

10.33 

11.47 

11.93 

<5  Yrs  to  Go 

% 

9.01 

7.58 

8.07 

8.09 

7.61 

6.49 

6.64 

6.14 

7.26 

7.29 

Ret  Eligible 

% 

5.48 

3.84 

4.33 

4.48 

4.05 

3.24 

3.32 

2.66 

3.02 

3.53 

LOSS  FROM  DOD  CIV 

No 

% 

49.42 

54.14 

56.16 

59.85 

62.81 

72.81 

76.07 

77.82 

79.78 

81.83 

Yes 

% 

50.58 

45.86 

43.84 

40.15 

37.19 

27.19 

23.93 

22.18 

20.22 

18.17 

LOSS  FROM  CIV  DAW 

No  loss 

% 

27.3 

33.28 

32.37 

31.74 

38.81 

54.34 

57.82 

63.38 

66.93 

68.58 

Administrative 

% 

21.38 

16.59 

17.1 

22.84 

17.54 

8.91 

8.95 

6.56 

5.95 

6.09 

Substantive 

% 

18.39 

17.8 

19.88 

18.44 

17.1 

14.68 

13.52 

10.73 

8.77 

8.97 

Separation 

% 

32.93 

32.33 

30.65 

26.97 

26.55 

22.08 

19.71 

19.33 

18.36 

16.37 

TENURE  DOD  CIV 

Mean 

10 

9 

8 

8 

7 

6 

5 

4 

4 

3 

TENURE  CIV  DAW 

Mean 

7 

7 

6 

5 

5 

5 

4 

4 

3 

2 

a  Although  the  mean  for  COMPENSATION  is  displayed  in  this  table,  the  propensity-score-based  weighting  procedures  and  the 
Cox  PH  models  in  this  study  use  the  log  of  COMPENSATION  (In(COMPENSATION)) 

b  This  category  includes  individuals  in  U.S.  territories  that  are  not  states,  individuals  in  foreign  locations,  and  individuals  for 
whom  location  information  is  not  recorded 
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Chapter  7.  Descriptive  Analysis 


The  goal  of  the  descriptive  analysis  is  to  explore  the  past  work  experience  data  files 
described  in  the  previous  chapter  to  address  the  first  research  question:  How  did  the  growth 
initiative  change  who  the  civilian  DAW  hired  into  its  ranks,  in  terms  of  prior  work  experience? 
As  described  in  Chapter  5,  one  should  expect  to  see  a  larger  percentage  of  entrants  with  no  prior 
DoD  experience  in  the  growth  initiative  cohorts,  compared  to  the  cohorts  prior  to  FY  2008. 

The  descriptive  analysis  examines  the  data  from  a  number  of  perspectives.  First,  the  analysis 
describes  the  cohorts  in  terms  of  where  the  entrants  were  in  the  FY  prior  to  the  year  they  joined 
the  civilian  DAW.  After  that,  the  analysis  describes  each  cohort  in  terms  of  the  entrants’  overall 
prior  DoD  experience,  reaching  back  to  FY  1980. 

In  the  end,  the  evidence  supports  the  hypothesis:  The  hiring  surge  cohorts  have  a 
significantly  higher  percentage  of  outside  hires  with  no  prior  DoD  experience. 

The  Growth  Initiative  was  Fueled  Mainly  by  Outside  Hires  with  No  Prior 
Work  Experience  in  the  DoD 

Based  on  the  methodology  described  above,  each  civilian  DAW  entrant  from  FY  2000  to  FY 
2014  included  in  this  analysis  is  marked  as  in  one  of  three  workforces  in  each  FY  from  1980  to 
1991  and  in  one  of  five  workforces  from  1992  until  he  or  she  enters  the  civilian  DAW  (see 
Figures  6.3  and  6.4).  This  allows  one  to  examine  which  workforce  an  entrant  was  in  in  the  FY 
before  he  or  she  entered  the  civilian  DAW,  shedding  light  on  where  (which  workforces)  civilian 
DAW  entrants  tend  to  come  from. 

Figure  7.1,  below,  shows  where  civilian  DAW  entrants  have  come  from  over  the  past  15 
years.  The  graphic  shows  that  the  cohort-size  increase  during  the  years  of  the  DAW  hiring  surge 
was  almost  entirely  fueled  by  hires  from  outside  of  the  DoD.  During  the  four  years  of  the  DAW 
growth  initiative  (FYs  2008-2011),  a  total  of  41,853  individuals  joined  the  civilian  DAW  from  a 
position  outside  of  the  DoD;  more  than  14,000  of  those  individuals  were  hired  in  FY  2010  alone. 
In  contrast,  a  total  of  47,160  individuals  joined  the  civilian  DAW  from  a  position  outside  of  the 
DoD  during  the  1 1  non-growth-initiative  years  included  in  this  analysis  (FYs  2000-2007  and 
2012-2014),  with  no  more  than  5,700  individuals  coming  from  an  outside-of-the-DoD  position  in 
any  given  FY. 

Figure  7.1  illustrates  that  the  number  of  civilian  DAW  entrants  coming  from  a  non¬ 
acquisition  DoD  civilian  position  has  remained  relatively  constant  for  every  cohort  examined  in 
this  analysis,  including  those  cohorts  during  the  DAW  hiring  surge  years.  Specifically,  no  cohort 
has  fewer  than  2,100  or  more  than  5,800  entrants  coming  from  a  non-acquisition  DoD  civilian 
position. 
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Figure  7.1.  DAW  Hiring  Surge  was  Fueled  by  Hires  from  Outside  of  the  DoD 
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DAW) 


FY  Cohort 


DAW  Hiring  Surge 


Individuals  who  join  the  civilian  DAW  from  a  position  that  is  outside  of  the  DoD  could  still 
have  some  DoD  experience  in  previous  years.  In  the  end,  the  data  reveal  that  although  many 
outside-of-the-DoD  hires  have  some  prior  DoD  experience,  the  majority  of  them  do  not. 

Figure  7.2  shows  the  number  of  outside-of-the-DoD  hires  in  each  cohort  who  do  and  do  not 
have  prior  DoD  experience  upon  entering  the  civilian  DAW.  The  graphic  shows  that  hires  from 
outside  of  the  DoD  are  more  likely  to  have  no  prior  DoD  experience  when  they  enter  the 
workforce.  This  trend  holds  for  every  cohort  examined — even  for  the  larger  cohorts  during  the 
DAW  growth  initiative.  In  fact,  the  percentage  of  outside-of-the-DoD  hires  without  prior  DoD 
experience  remains  relatively  stable  for  every  cohort,  hovering  between  53  and  64  percent  every 
FY,  even  though  the  overall  cohort  size  increased  substantially  in  response  to  the  growth 
initiative. 
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Figure  7.2.  Most  Individuals  Hired  into  the  Civilian  DAW  Directly  from  Positions  Outside  of  the 

DoD  Have  No  Prior  DoD  Work  Experience 
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The  Fraction  of  Entrants  with  the  DoD  Civilian  Prior  Career  Profile 
Decreased  During  the  Hiring  Surge 

The  following  figures  divide  each  cohort  into  mutually  exclusive  prior-work-experience- 
combination  groups,  or  “prior  career  profiles.”  These  profiles  are  listed  and  described  in  Table 
7.1.  The  first  five  profiles  are  the  top  five  most  common  prior  DoD  experience  combinations. 
Entrants  with  other  past  DoD  experience  combinations  fall  into  the  “Other  DoD  Experience” 
profile,  and  entrants  with  no  prior  DoD  experience  fall  into  the  “No  DoD  Experience”  profile. 
Each  entrant  for  all  cohorts  examined  is  categorized  into  only  one  of  these  profiles. 
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Table  7.1.  Prior  Career  Profiles 


Prior  Career  Profile  An  Entrant  is  Categorized  into  this  Profile  When... 


DoD  Civilian 

DoD  Civilian  +  Civilian  DAW 
Active  Duty 

Active  Duty  +  DoD  Civilian 

Active  Duty  +  Military  DAW 

Other  DoD  Experience 
No  DoD  Experience 


The  only  DoD  experience  an  individual  has  is  in  the  DoD  civilian  workforce  (in  a 
non-DAW  position) 

An  individual  has  had  experience  in  both  the  DoD  civilian  workforce  and  in  the 
civilian  DAW  but  has  no  other  type  of  DoD  experience 

The  only  DoD  experience  an  individual  has  is  in  the  active  duty  military  (in  a 
non-DAW  position) 

An  individual  has  had  experience  in  both  the  active  duty  military  (in  a  non-DAW 
position)  and  in  the  DoD  civilian  workforce  (in  a  non-DAW  position)  but  has  no 
other  type  of  DoD  experience 

An  individual  has  been  in  the  active  duty  military  in  both  an  acquisition  position 
and  a  non-acquisition  position  but  has  no  other  type  of  DoD  experience 

An  individual  has  any  other  prior  DoD  experience  combination 

An  individual  has  no  prior  DoD  work  experience 


Figure  7.3,  below,  shows  how  the  number  of  entrants  with  each  of  these  prior  career  profiles 
changes  from  FY  2000  until  FY  2014.  Each  line  represents  the  number  of  entrants  in  each  cohort 
with  that  particular  profile.  The  sum  of  all  lines  for  a  given  cohort  year  is  the  total  number  of 
entrants  in  each  FY  cohort.  The  chart  illustrates  that  the  civilian  DAW  growth  initiative  was 
mainly  fueled  by  outside  hires  with  no  prior  DoD  experience.  There  is  also  an  increase — 
although  not  as  drastic — in  the  number  of  entrants  with  active  duty  (non-DAW)  experience.  This 
is  because  outside  hires  with  DoD  experience  tend  to  have  active  duty  military  experience. 

Figure  7.4  shows  the  same  data  but  in  percentage  tenns  so  that  the  prior  career  profiles  can 
be  compared  more  easily  across  FY  cohorts.  This  chart  highlights  that  there  is  a  significant 
decrease  in  the  percentage  of  entrants  with  the  DoD  civilian  prior  career  profile  and  an  increase 
in  the  percentage  of  hires  with  no  prior  DoD  experience  during  the  hiring  surge  years.  Also,  note 
that  the  percentage  of  entrants  who  join  the  civilian  DAW  with  active  duty  experience  does  not 
change  significantly  during  the  growth  initiative  years.  In  fact,  this  line  suggests  that  the 
percentage  of  entrants  with  active  duty  experience  has  been  slowly  increasing  over  these  15 
cohort  years. 
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Number  of  Entrants 


Figure  7.3.  Civilian  DAW  Growth  was  Mainly  Fueled  by  Outside  Hires  with  No  Prior  DoD 
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Figure  7.4.  The  Hiring  Surge  Saw  a  Decrease  in  the  Percentage  of  Entrants  with  the  DoD  Civilian 
Prior  Career  Profile  and  an  Increase  in  the  Percentage  of  Entrants  with  No  Prior  DoD  Experience 
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In  the  end,  the  analysis  supports  the  hypothesis.  The  hiring  surge  was  mostly  fueled  by  hires 
from  outside  of  the  DoD,  and  the  majority  of  these  outside  hires  have  never  been  official  DoD 
employees.  The  analysis  shows  that  the  growth  initiative  cohorts  have  a  greater  percentage  of 
outside  hires  with  no  prior  DoD  experience  and  a  lower  fraction  of  entrants  with  the  DoD 
civilian  prior  career  profile.  This  shift  in  cohort  characteristics  brought  about  by  the  growth 
initiative  could  have  overall  retention  implications.  The  following  chapters  explore  the 
relationship  between  prior  work  experience  and  retention. 
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Chapter  8.  Retention  Analysis  Part  I:  How  DoD  Experience 
Relates  to  Retention 


The  following  analysis  utilizes  the  retention  data  and  survival-analysis  techniques  to  address 
the  second  research  question:  How  does  entrants’  prior  work  experience  relate  to  retention?  To 
address  this  question,  this  retention  analysis  focuses  on  the  pre-policy  cohorts  because  tests  on 
these  earlier  cohorts  capture  longer-term  retention  trends.  The  analysis  utilizes  the  retention 
analysis  data  file  outlined  in  Table  6.4. 

To  check  robustness,  the  analysis  runs  separate  tests  on  three  different  pre -policy  cohort 
groups.  These  cohort  groups  are  outlined  in  Table  8.1.  The  “early  pre-policy  cohort  group” 
consists  of  FY  cohorts  2001-2003.  The  “later  pre -policy  cohort  group”  consists  of  FY  cohorts 
2004,  2005,  and  2007. 50  And  the  “combination  pre-policy  cohort  group”  consists  of  all  of  the 
available  pre-policy  cohorts  (FY  cohorts  2001-2005  and  2007).  The  retention  analysis  described 
in  this  chapter  includes  tests  on  all  three  of  these  pre-policy  cohort  groups.  A  robust  finding  is 
one  that  holds  across  all  three  samples. 

Table  8.1.  Pre-Policy  Cohort  Groups  for  Analysis 

Cohort  Group  Name  FY  Cohorts  Included 

Early  pre-policy  cohort  group  2001-2003 

Later  pre-policy  cohort  group  2004,  2005,  and  2007 

Combination  pre-policy  cohort  group  2001-2005  and  2007 

This  study  uses  Cox  Proportional  Hazard  (Cox  PH)  regression  models  to  statistically  test  for 
differences  in  retention  trends  across  different  prior  work  experience  groups.  The  Cox  PH  model 
produces  an  estimate  called  the  hazard  ratio.  For  the  purposes  of  the  following  analyses,  the 
hazard  ratio  estimates  the  probability  that  individuals  with  certain  prior  work  experiences 
experience  a  loss,  i.e.,  substantively  transfer  out  of  or  separate  from  the  civilian  DAW,  in  any 
given  year  in  the  workforce  relative  to  a  reference  group  (Allison,  2010;  Guo,  Hall-Partyka  and 
Gates,  2014).  A  hazard  ratio  estimate  that  is  greater  than  one,  say  1.06,  indicates  that  that 
particular  group  of  individuals  is  more  likely  (specifically,  six  percent  more  likely)  than  the 
reference  group  to  experience  a  loss  in  any  given  year  in  the  workforce  (i.e.,  they  have  a  greater 
“hazard”  of  experiencing  a  loss).  On  the  contrary,  a  hazard  ratio  that  is  less  than  one,  say  0.90, 
indicates  that  that  particular  group  of  individuals  is  less  likely  (specifically,  ten  percent  less 


50  As  described  in  Chapter  6,  the  FY  2006  cohort  is  excluded  because  a  substantial  percentage  of  individuals  in  the 
FY  2006  civilian  DAW  cohort  has  missing  information. 
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likely)  than  the  reference  group  to  experience  a  loss  in  any  given  year  in  the  workforce  (i.e.,  they 
have  a  lower  “hazard”  of  experiencing  a  loss)  (Allison,  2010,  p.  132;  Guo,  Hall-Partyka  and 
Gates,  2014,  p.  13). 

Individuals  who  join  the  civilian  DAW  with  different  prior  work  experiences  also  tend  to 
differ  across  the  other  explanatory  variables  listed  in  Table  6.2,  and  these  observable 
characteristics  may  also  influence  retention.  Therefore,  in  order  to  tease  out  the  true  relationship 
between  prior  work  experience  and  retention,  this  analysis  uses  two  methods  to  control  for  these 
differences  in  characteristics:  (1)  propensity-score -based  weighting  and  (2)  “multivariate 
adjustment”51  techniques.52  Propensity-score-based  weighting  balances  the  “comparison 
group[s]”  (e.g.,  entrants  with  prior  DoD  experience  and  entrants  with  no  prior  DoD  experience) 
on  the  observable  characteristics  using  weights.  The  weights  emphasize  certain  individuals  but 
deemphasize  others  in  each  group  in  order  to  make  each  comparison  group  look  more  like  the 
overall  average  observation,  based  on  all  of  the  available  descriptors  (Griffin  and  McCaffrey, 
2015a;  Griffin  and  McCaffrey,  2015b;  McCaffrey,  Ridgeway  and  Morral,  2004).  Multivariate 
adjustment  involves  including  the  relevant  covariates  in  the  Cox  PH  models  (Griffin  and 
McCaffrey,  20 1 5d). 

Tests  comparing  retention  trends  between  only  two  groups  (e.g.,  entrants  with  prior  DoD 
experience  vs.  entrants  with  no  prior  DoD  experience)  use  these  two  control  techniques 
together — to  produce  what  are  called  “doubly  robust”54  estimates  (Griffin  and  McCaffrey, 

2015b;  Griffin  and  McCaffrey,  2015d;  Griffin  and  McCaffrey,  2015c,  p.  41;  Griffin  and 
McCaffrey,  2015a,  p.  45).  Griffin  and  McCaffrey  (2015d)  state  that  doubly  robust  models  are  the 
“gold  standard”  in  tenns  of  adjusting  for  observable  confounding  factors. 

It  is  not  possible  to  use  doubly  robust  Cox  PH  models  to  test  for  differences  in  retention 
trends  across  the  different  prior  career  profiles  outlined  in  Table  7.1.  The  individuals  in  certain 
prior  career  profile  groups  are  too  fundamentally  different,  in  terms  of  observable  characteristics, 
from  individuals  with  no  prior  DoD  experience  (the  reference  group) — applied  propensity-score 
weights  cannot  bring  these  groups  into  balance.  This  highlights  the  fact  that  there  is  not  much 
“overlap”  (Griffin  and  McCaffrey,  2015a,  p.  29;  Griffin  and  McCaffrey,  2015b)  between 
individuals  with  certain  prior  career  profiles  that  include  DoD  experience  and  non-DoD- 
experienced  entrants,  weakening  the  argument  for  causality.  Given  this,  the  results  from  the  non- 
doubly-robust  models  (i.e.,  the  models  that  test  for  retention  differences  across  the  various  prior 
career  profiles  and  that  incorporate  only  multivariate  adjustment)  are  interpreted  conservatively, 


51  Griffin  and  McCaffrey  (2015d)  use  the  term  “multivariate  adjustment.”  This  study  uses  this  same  term. 

52 

See  Appendix  B  for  more  detailed  information  on  these  methods. 

53  McCaffrey,  Ridgeway  and  Morral  (2004)  (p.  404),  Griffin  and  McCaffrey  (2015a)  (p.  36),  and  Griffin  and 
McCaffrey  (2015b)  use  the  phrase  “comparison  group”  in  their  work  on  propensity-score  weighting.  This  analysis 
uses  the  same  phrase. 

54  The  term  “doubly  robust”  stems  from  Griffin  and  McCaffrey  (2015b),  Griffin  and  McCaffrey  (2015d),  Griffin 
and  McCaffrey  (2015c)  (p.  41),  Griffin  and  McCaffrey  (2015a)  (p.  45).  This  analysis  also  uses  this  term. 
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focusing  on  the  general  direction  of  the  hazard  ratio  estimates  rather  than  on  the  precise  estimate 
numbers. 

As  described  in  Chapter  6,  there  are  three  types  of  losses  from  the  civilian  DAW:  (1) 
administrative  transfers  out,  (2)  substantive  transfers  out,  and  (3)  separations.  The  following 
analyses  focus  on  substantive  transfers  out  and  separations;  administrative  transfers  out  are 
censored  in  all  cases  since  these  represent  bureaucratic  changes  rather  than  actual  personnel 
movement.  In  other  words,  the  term  “retention”  (in  regard  to  retention  in  the  civilian  DAW) 
refers  to  an  individual’s  hazard  of  substantively  transferring  out  of  or  separating  from  the  civilian 
DAW.  Individuals  with  a  lower  hazard  of  substantively  transferring  out  of  or  separating  from  the 
civilian  DAW  (i.e.,  those  who  are  less  likely  to  substantively  transfer  out  or  separate)  are  said  to 
have  greater  retention.  Although  the  focus  of  this  analysis  is  on  this  definition  of  retention,  which 
includes  both  uncensored  loss  types — substantive  transfers  out  and  separations — the  analysis 
also  examines  these  loss  types  individually  in  order  to  further  explore  the  relationship  between 
prior  work  experience  and  retention. 

There  is  No  Evidence  Suggesting  that  Prior  DoD  Experience  Leads  to 
Greater  Retention  in  the  Civilian  DAW 

Entrants  with  prior  DoD  experience  are,  on  average,  fundamentally  different  from  entrants 
with  no  prior  DoD  experience.  Tables  C.l.l-C.3.1  in  Appendix  C  illustrate  how  the  two  groups 
balance  in  terms  of  the  other  observable  characteristics  included  in  this  study.  Specifically,  the 
tables  show  that,  among  other  things,  DoD-experienced  entrants  tend  to  be  older  and — in 
general — closer  to  retirement  eligibility  relative  to  entrants  with  no  prior  DoD  experience. 

To  control  for  these  systematic  differences  between  entrants  with  and  without  DoD 
experience,  the  following  tests  include  propensity  score  weights55  and  multivariate  controls.  The 
doubly  robust  Cox  PH  models  show  that — all  else  equal — there  is  no  statistically  significant 
difference  in  retention  trends  between  entrants  with  prior  DoD  experience  and  those  with  no 
prior  DoD  experience.  Table  8.2  shows  these  results. 


55  Appendix  C  shows  that  the  applied  propensity-score  weights  significantly  improve  the  balance  between  the  two 
groups  (i.e.,  entrants  with  prior  DoD  experience  and  entrants  with  no  prior  DoD  experience),  based  on  the 
characteristics  listed  in  Table  6.2. 
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Table  8.2.  All  Else  Equal,  There  is  No  Evidence  Suggesting  that  Entrants  with  prior  DoD 
Experience  Have  a  Lower  Hazard  of  Substantively  Transferring  Out  of  or  Separating  From  the 
Civilian  DAW  Compared  to  Entrants  with  No  Prior  DoD  Experience 


Doubly  Robust  Cox  ph  Pre-Policy  Cohort  Group 


Early 

Later 

Combination 

N 

27224 

28643 

55867 

Degrees  of  Freedom 

27223 

28642 

55866 

Estimation 

-0.024 

0.028 

0.014 

Hazard  Ratio 

0.976 

1.028 

1.014 

Standard  Error 

0.057 

0.044 

0.033 

p- value 

0.6705 

0.5334 

0.6755 

Some  Evidence  Suggests  that  Internal  Hires  Have  Lower  Civilian  DAW 
Retention  than  External  Hires 

One  can  also  examine  how  each  entrant’s  most  recent  work  experience  relates  to  retention 
outcomes.  Specifically,  as  shown  in  Figure  7.1,  there  are  a  number  of  entrants  who  join  the 
civilian  DAW  directly  from  a  non-DAW  DoD  civilian  position.  Hereafter,  these  individuals  are 
referred  to  as  “internal  hires.”  Those  who  join  the  civilian  DAW  from  another  workforce — from 
the  military  DAW,  the  wider  active-duty  military,  or  somewhere  outside  the  DoD — are  hereafter 
referred  to  as  “external  hires.”56  As  depicted  in  Figure  7.1,  most  of  the  external  hires  are  outside 
hires  (i.e.,  they  come  from  a  position  outside  of  the  DoD  entirely).  The  following  analyses 
examine  how  civilian  DAW  retention  trends  may  differ  between  internal  and  external  hires. 

The  doubly  robust  Cox  PH  models57  show  some  evidence  that  internal  hires  have  lower 
retention  in  the  civilian  DAW  than  external  hires.'  These  results  are  shown  in  Table  8.3,  below. 


56  “External  hires”  are  similar  to  “new  hires,”  and  “internal  hires”  are  similar  to  “substantive  transfers”  as  defined  in 
the  “Defining  a  Cohort”  section  in  Chapter  6.  However,  the  “external  hire”  and  “internal  hire”  definitions  are  based 
on  the  past-work-experience  characterization  (specifically,  which  workforce  each  entrant  was  in  in  the  FY  prior  to 
him  or  her  joining  the  civilian  DAW),  which  adjusts  for  “double-count”  years  (FYs  in  which  one  individual  appears 
in  both  the  active-duty  data  file  and  the  DoD  civilian  data  file).  Therefore,  some  individuals  who  are  included  in  a 
cohort  as  a  transfer  are  actually  counted  as  external  hires.  This,  however,  is  not  common — there  are  only  10 
instances  where  this  occurs.  See  Appendix  I  for  more  information. 

57 

Appendix  D  shows  that  the  applied  propensity-score  weights  significantly  improve  the  balance  between  the  two 
groups  (i.e.,  internal  hires  and  external  hires),  based  on  the  characteristics  listed  in  Table  6.2. 
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This  is  counter  to  what  Guo,  Hall-Partyka  and  Gates  (2014)  find.  As  outlined  in  Chapter  3,  Guo,  Hall-Partyka  and 
Gates  (2014)  find  that  “Employees  who  transferred  into  the  [civilian  DAW]”  from  a  non-DAW  DoD  civilian 
position  have  greater  retention  than  employees  who  came  from  other  sources  (pp.  14,  22).  The  models  used  by  Guo, 
Hall-Partyka  and  Gates  (2014)  differ  from  those  used  in  this  study  because  the  analyses  address  different  questions. 
Guo,  Hall-Partyka  and  Gates  (2014)  ask  how  “personnel  quality,”  measured  by  “education  and  performance 
ratings,”  relates  to  civilian  DAW  retention  (p.  xi).  This  study  asks  how  past  work  experience  relates  to  civilian 
DAW  retention.  As  such,  Guo,  Hall-Partyka  and  Gates  (2014)  include  performance  ratings,  another  workforce 
outcome,  as  an  explanatory  variable;  this  study  does  not.  This  analysis  is  not  interested  in  understanding  how  prior 
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Specifically,  the  analysis  on  the  early  pre-policy  cohort  groups  fails  to  show  a  difference  in 
retention  trends  between  the  two  groups,  but  tests  on  the  later  and  combination  pre-policy  cohort 
groups  show  that  internal  hires  are  six  to  eight  percent  more  likely  to  substantively  transfer  out  of 
or  separate  from  the  civilian  DAW  in  any  given  year  relative  to  external  hires. 


Table  8.3.  There  is  Some  Evidence  Suggesting  that  Internal  Hires  Have  a  Greater  Hazard  of 
Substantively  Transferring  Out  of  or  Separating  From  the  Civilian  DAW  Compared  to  External 

Hires 


Doubly  Robust  cox  ph  Pre-Policy  Cohort  Group 


Early 

Later 

Combination 

N 

27224 

28643 

55867 

Degrees  of  Freedom 

27223 

28642 

55866 

Estimation 

0.038 

0.076 

0.059 

Hazard  Ratio 

1.039 

1.079 

1.061 

Standard  Error 

0.034 

0.028 

0.020 

p-value 

0.2583 

0.0065 

0.0030 

Although  the  tests  suggest  that  internal  hires  may  have  lower  levels  of  overall  civilian  DAW 
retention,  analyses  that  examine  each  loss  type  individually  show  that  internal  hires  are  actually 
less  likely  than  external  hires  to  separate  from  the  civilian  DAW  but  much  more  likely  to 
substantively  transfer  out  of  the  civilian  DAW.  These  results  are  shown  in  Tables  8.4  and  8.5, 
below.  According  to  these  doubly  robust  Cox  PH  models,  internal  hires  are  approximately  30-42 
percent  more  likely  to  substantively  transfer  out  of  the  civilian  DAW  in  any  given  year  relative 
to  external  hires.  On  the  contrary,  the  models  estimate  that  internal  hires  are  approximately  eight 
to  1 1  percent  less  likely  to  separate  from  the  civilian  DAW  in  any  given  year  relative  to  external 
hires. 


work  experience  relates  to  performance  and  retention  separately;  rather,  it  is  interested  in  understanding  how  past 
work  experience  relates  to  retention,  regardless  of  how  retention  and  performance  interact.  This  difference  in  model 
specification,  along  with  other  differences,  may  play  a  role  in  these  contrasting  estimates. 
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Table  8.4.  Internal  Hires  Have  a  Greater  Hazard  of  Substantively  Transferring  Out  of  the  Civilian 

DAW  Relative  to  External  Hires 


Doubly  Robust  Cox  PH 

Pre-Policy  Cohort  Group 

Early 

Later 

Combination 

N 

27224 

28643 

55867 

Degrees  of  Freedom 

27223 

28642 

55866 

Estimation 

0.262 

0.351 

0.315 

Hazard  Ratio 

1.300 

1.421 

1.371 

Standard  Error 

0.057 

0.048 

0.035 

p-value 

<.0001 

<.0001 

<.0001 

Table  8.5.  Internal  Hires  Have  a  Lower  Hazard  of  Separating  From  the  Civilian  DAW  Compared  to 

External  Hires 


Doubly  Robust  Cox  ph  Pre-Policy  Cohort  Group 


Early 

Later 

Combination 

N 

27224 

28643 

55867 

Degrees  of  Freedom 

27223 

28642 

55866 

Estimation 

-0.086 

-0.120 

-0.098 

Hazard  Ratio 

0.918 

0.887 

0.906 

Standard  Error 

0.042 

0.036 

0.025 

p-value 

0.0404 

0.0008 

<.0001 

In  the  end,  these  analyses  show  some  evidence  that  entrants  who  join  the  civilian  DAW 
directly  from  a  non-DAW  DoD  civilian  position  have  lower  rates  of  retention  than  other  civilian 
DAW  entrants.  And  the  tests  that  focus  on  substantive  transfers  out  and  separations  individually 
suggest  that  internal  hires  are  more  attached  to  the  DoD  civilian  workforce  in  general  than  are 
external  hires.  Analyses  on  retention  in  the  wider  DoD  civilian  workforce  support  this  finding: 
Internal  hires  have  higher  retention  trends  in  the  wider  DoD  civilian  workforce  than  external 
hires.  The  results  for  these  tests  on  retention  differences  in  the  wider  DoD  civilian  workforce  are 
shown  in  Table  F.l  in  Appendix  F. 

Among  External  Hires,  Those  with  Prior  DoD  Experience  Tend  to  Have 
Lower  Civilian  DAW  Retention 

The  following  analyses  estimate  how  retention  trends  vary  across  prior  work  experiences  for 
the  external-hire  population. 

The  first  tests  estimate  how  civilian  DAW  retention  differs  between  external  hires  with  prior 
DoD  experience  and  those  with  no  prior  DoD  experience.  The  results  from  the  doubly  robust 
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Cox  PH  models59  are  shown  in  Table  8.6,  below.  The  models  show  that — among  external 
hires — entrants  with  prior  DoD  experience  tend  to  have  lower  retention  than  those  with  no  prior 
DoD  experience.  Specifically,  the  models  estimate  that  external  hires  with  prior  DoD  experience 
are  13-15  percent  more  likely  to  substantively  transfer  out  of  or  separate  from  the  civilian  DAW 
in  any  given  year  compared  to  external  hires  with  no  prior  DoD  experience. 

Table  8.6.  Among  External  Hires,  Those  with  Prior  DoD  Experience  Have  a  Greater  Hazard  of 
Substantively  Transferring  Out  of  or  Separating  From  the  Civilian  DAW  Compared  to  Those  with 

No  Prior  DoD  Experience 


Doubly  Robust  Cox  ph  Pre-Policy  Cohort  Group 


Early 

Later 

Combination 

N 

12471 

14578 

27049 

Degrees  of  Freedom 

12470 

14577 

27048 

Estimation 

0.141 

0.126 

0.119 

Hazard  Ratio 

1.151 

1.134 

1.126 

Standard  Error 

0.051 

0.055 

0.038 

p-value 

0.0053 

0.0210 

0.0018 

“DoD  experience”  is  a  broad  term:  Individuals  could  have  a  number  of  different  prior-work- 
experience  combinations  within  the  DoD,  and  retention  trends  could  vary  across  these  different 
experience  profiles.  The  following  analysis  estimates  how  the  various  prior  career  profiles  relate 
to  civilian  DAW  retention  patterns,  focusing  solely  on  external  hires.  These  models  are  not 
“doubly  robust” — they  incorporate  multivariate  controls  but  not  propensity  score  weights.  The 
results  are  interpreted  generally,  not  specifically,  focusing  on  the  general  direction  of  the  hazard 
ratio  estimates,  not  on  the  precise  estimate  number. 

The  Cox  PH  models  show  that  external  hires  with  the  DoD  Civilian  prior  career  profile  have 
lower  civilian  DAW  retention  than  external  hires  with  no  prior  DoD  experience.  The  models 
estimate  the  same  relationship  for  external  hires  with  the  Active  Duty,  Active  Duty  +  DoD 
Civilian,  or  Other  prior  career  profile,  and  there  is  some  evidence  that  this  is  also  true  for 
external  hires  with  the  DoD  Civilian  +  Civilian  DAW  prior  career  profile.  External  hires  with  the 
Active  Duty  +  Military  DAW  prior  career  profile  exhibit  significantly  greater  retention  trends  in 
the  analysis  on  the  early  pre-policy  cohort  group  and  significantly  lower  retention  trends  in  the 
analysis  on  the  later  pre-policy  cohort  group.  The  results  for  these  tests  are  displayed  in  Table 
8.7.  In  summary,  there  is  some  evidence  suggesting  that  external  hires  with  any  of  the  prior 
career  profiles  that  include  DoD  experience  (with  the  exception  of  those  with  the  Active  Duty  + 
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Appendix  E  shows  that  the  applied  propensity-score  weights  significantly  improve  the  balance  between  the  two 
groups  (i.e.,  external  hires  with  no  prior  DoD  experience  and  external  hires  with  prior  DoD  experience),  based  on 
the  characteristics  listed  in  Table  6.2. 
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Military  DAW  prior  career  profile,  who  also  exhibit  greater  retention  in  one  analysis)  have  lower 
retention  than  external  hires  with  no  prior  DoD  experience.60 


Table  8.7.  In  General,  External  Hires  with  a  Prior  Career  Profile  that  Includes  DoD  Experience  (With 
the  Exception  of  the  Active  Duty  +  Military  DAW  Prior  Career  Profile)  Have  a  Greater  Hazard  of 
Substantively  Transferring  Out  of  or  Separating  From  the  Civilian  DAW  Compared  to  External 

Hires  with  No  Prior  DoD  Experience 


Cox  PH  with  Controls 

Hazard  Ratio  Estimation  (ref=‘ 

‘no  prior  DoD  experience”) 

Prior  Career  Profile 

Early 

%  External 

Later 

%  External 

Combination 
%  External 

Hires 

Estimate 

Hires 

Estimate 

Hires 

Estimate 

DoD  Civ 

8.36 

“*1.186 

5.76 

“1.147 

6.95 

***1.167 

DoD  Civ  +  Civ  DAW 

5.97 

1.101 

4.23 

“1.159 

5.03 

“1.123 

Active  Duty 

23.4 

***1.178 

24.34 

“1.108 

23.91 

***1.139 

Active  Duty  +  DoD  Civ 

1.91 

***1.334 

1.62 

***1.379 

1.75 

***1.363 

Other  DoD  Experience 

1.76 

***1.482 

2.15 

***1.383 

1.97 

***1.43 

Active  Duty  +  Mil  DAW 

5.82 

*1.126 

8.18 

“0.865 

7.09 

0.977 

p-value<  *0.1,  **0.05,  “*0.01 


In  the  end,  the  estimates  displayed  above  show  that — in  general61 — external  hires  with  prior 
DoD  experience  tend  to  have  lower  retention  rates  in  the  civilian  DAW  than  external  hires  with 
no  prior  DoD  experience. 

The  Results  Fail  to  Support  the  Hypothesis 

The  retention  analysis  fails  to  support  the  original  hypothesis — that  entrants  with  prior  DoD 
experience  would  have  greater  retention  in  the  civilian  DAW  than  entrants  with  no  prior  DoD 
experience.  In  fact,  some  results  suggest  the  opposite  relationship.  Specifically,  the  analyses  find 
some  evidence  that  internal  hires  have  lower  civilian  DAW  retention  rates  than  external  hires. 
And  additional  tests  find  that,  in  general,  external  hires  with  no  prior  DoD  experience  tend  to 
have  the  greatest  civilian  DAW  retention. 

The  tests  that  examine  substantive  transfers  out  and  separations  individually,  as  well  as  tests 
on  retention  in  the  wider  DoD  civilian  workforce,  shed  some  light  on  the  potential  mechanisms 
behind  these  unexpected  findings.  In  particular,  the  tests  show  that  internal  hires  are  significantly 
more  likely  than  external  hires  to  substantively  transfer  out  of  the  civilian  DAW  but  significantly 
less  likely  to  separate  from  the  civilian  DAW  in  any  given  year.  In  addition,  internal  hires  are 


60  The  sensitivity  analysis  in  Appendix  H  shows  that  when  the  “lag  instances”  are  not  included  in  the  civilian  DAW 
cohorts,  the  tests  on  the  early  and  later  pre-policy  cohort  groups  between  external  hires  with  the  Active  Duty  + 
Military  DAW  prior  career  profile  and  external  hires  with  no  prior  DoD  experience  find  no  statistically  significant 
difference  in  retention  trends  (see  Table  H.1.7). 

61  With  the  exception  of  external  hires  with  the  Active  Duty  +  Military  DAW  prior  career  profile 
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found  to  have  greater  retention  in  the  wider  DoD  civilian  workforce  than  external  hires.  These 
findings  suggest  that  internal  hires  are  much  more  comfortable,  relative  to  external  hires,  moving 
from  a  civilian  DAW  position  to  a  position  in  the  wider  DoD  civilian  workforce. 
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Chapter  9.  Retention  Analysis  Part  II:  How  the  Growth  Initiative 
Relates  to  Retention 


The  second  part  of  the  retention  analysis  addresses  the  third  and  final  research  question. 
Specifically,  the  analysis  aims  to  evaluate  how  the  hiring  surge  may  have  influenced  retention  in 
the  civilian  DAW.  It  is  impossible  to  tell  exactly  what  would  have  happened  (in  terms  of 
retention  outcomes)  if  the  growth  initiative  had  not  happened,  but  the  following  analysis  uses  a 
synthetic-cohort  comparison  technique  to  make  an  estimate.  These  estimates  will  help 
acquisition  personnel  managers  understand  what  effect  the  DAW  hiring  surge  has  on  overall 
cohort-level  retention,  which  will  in  turn  help  inform  workforce  policy  decisions  in  the  future. 

The  following  analysis  utilizes  the  retention  analysis  data  file  described  in  Table  6.4.  The 
analysis  separates  the  available  FY  cohorts  (2001-2005,  2007-2011)  into  four  cohort  groups. 
Similar  to  the  analysis  described  in  Chapter  8,  the  early  pre-policy  cohort  group  consists  of  FY 
cohorts  2001-2003;  the  later  pre-policy  cohort  group  consists  of  FY  cohorts  2004,  2005,  and 
2007;  and  the  combination  pre-policy  cohort  group  consists  of  FY  cohorts  2001-2005  and  2007. 
The  policy  cohort  group  is  made  up  of  the  four  growth  initiative  years,  FY  cohorts  2008-2011. 
These  cohort  groups  are  displayed  in  Table  9.1. 

Table  9.1.  Pre-Policy  and  Policy  Cohort  Groups  for  Retention  Analysis,  Part  II62 


Cohort  Group  Name  FY  Cohorts  Included 

Early  pre-policy  cohort  group  2001-2003 

Later  pre-policy  cohort  group  2004,  2005,  and  2007 

Combination  pre-policy  cohort  group  2001-2005  and  2007 
Policy  cohort  group  2008-2011 

One  way  to  understand  how  the  growth  initiative  may  have  influenced  overall  retention 
trends  in  the  civilian  DAW  is  to  estimate  what  would  have  happened  (in  terms  of  retention 
outcomes)  if  the  policy  cohorts  had  looked  more  like  their  pre-policy  counterparts,  based  on  the 
available  observable  characteristics,  including  the  prior-career-profile  characteristics.  In  other 
words,  what  would  the  FY  2008-2011  cohort  group  retention  rates  have  been  if  the  growth 
initiative  had  not  occurred? 

To  address  this  question,  the  following  analysis  uses  a  synthetic-cohort  technique.  This 
involves  two  steps:  (1)  creating  a  counterfactual,  by  weighting  the  policy  cohort  group  to  look 
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“  As  described  in  Chapter  6,  the  FY  2006  cohort  is  excluded  because  a  substantial  percentage  of  individuals  in  the 
FY  2006  civilian  DAW  cohort  has  missing  information. 
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like  a  pre -policy  cohort  group,  based  on  most  of  the  characteristics  in  Table  6.263  and  on  the 
prior-career-profile  characteristics,  and  (2)  comparing  retention  trends  between  the  weighted 
policy  cohort  group  with  the  unweighted  policy  cohort  group.  Three  synthetic  cohort  groups  are 
created:  one  weighted  to  match  the  early  pre-policy  cohort  group  (hereafter  called  the  “synthetic 
early  pre-policy  cohort  group”),  one  weighted  to  match  the  later  pre-policy  cohort  group 
(hereafter  called  the  “synthetic  later  pre-policy  cohort  group”),  and  another  weighted  to  match 
the  combination  pre-policy  cohort  group  (hereafter  referred  to  as  the  “synthetic  combination  pre- 
policy  cohort  group”).64  After  creating  these  counterfactuals,  the  analysis  tests  for  differences  in 
retention  trends  between  the  unweighted  policy  cohort  group  and  each  of  the  synthetic  pre-policy 
cohort  groups.  Cox  PH  models  are  used  to  test  for  differences  in  retention.  The  models  include 
multivariate  controls  for  specific  cohort  year,  as  the  cohort  year  distribution  may  differ  between 
the  unweighted  policy  cohort  group  and  the  synthetic  pre-policy  cohort  groups.  Table  9.2,  below, 
outlines  the  three  comparison  tests.  Running  comparison  tests  using  all  three  synthetic  pre-policy 
cohort  groups  is  a  way  to  test  the  results’  robustness.  If  the  normal,  unweighted  policy  cohort 
group  tends  to  have  higher  rates  of  retention  than  the  synthetic  pre-policy  cohort  groups,  then 
one  would  conclude  that  the  growth  initiative  likely  increased  overall  levels  of  retention. 


Table  9.2.  To  Understand  What  Policy  Cohort  Retention  Trends  Would  Have  Been  had  the  Policy 
Cohorts  Looked  More  Like  Pre-Policy  Cohorts,  the  Analysis  Runs  the  Following  Comparisons 


Use  Cox  PH  models  to... 


1 .  Compare  retention  trends  from  the  policy  cohort  group  to  the  retention  trends  from  the  synthetic  early  pre¬ 
policy  cohort  group3 

2.  Compare  retention  trends  from  the  policy  cohort  group  to  the  retention  trends  from  the  synthetic  later  pre¬ 
policy  cohort  groupb 

3.  Compare  retention  trends  from  the  policy  cohort  group  to  the  retention  trends  from  the  synthetic 
combination  pre-policy  cohort  group3 

3 The  synthetic  early  pre-policy  cohort  group  is  the  policy  cohort  group  weighted  to  look  like  the  early  pre-policy 
cohort  group 

bThe  synthetic  later  pre-policy  cohort  group  is  the  policy  cohort  group  weighted  to  look  like  the  later  pre-policy 
cohort  group 

cThe  synthetic  combination  pre-policy  cohort  group  is  the  policy  cohort  group  weighted  to  look  like  the 
combination  pre-policy  cohort  group 
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The  following  characteristics  are  not  used  in  the  creation  of  the  synthetic  cohort  groups:  Cohort  Year  and 
LN(Compensation).  Cohort  Year  is  not  included  because  it  is  impossible  to  weight  one  cohort  group  to  match  the 
cohort  year  distribution  of  another  cohort  group.  LN(Compensation)  is  not  included  because,  in  general,  nominal 
compensation  levels  tend  to  increase  over  time,  so  it  would  be  inaccurate  to  weight  one  cohort  group  to  match  the 
compensation  distribution  of  another  cohort  group.  For  the  creation  of  the  synthetic  cohorts,  the  Broad  GS  Pay  Plan 
group  and  the  NSPS  Pay  Plan  group  are  combined  into  one  group,  which  is  called  “Broad  GS  or  NSPS.” 
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Appendix  G  explains  in  more  detail  how  these  synthetic  cohort  groups  are  created.  Appendix  G  also  shows  that 
the  weighting  schemes  used  to  create  the  synthetic  pre-policy  cohort  groups  are  successful — the  variable  means 
from  the  synthetic  pre-policy  cohort  groups  resemble  those  from  the  actual  pre -policy  cohort  groups. 
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The  policy  cohorts  and  pre-policy  cohorts  differ  across  several  dimensions,  not  just  in  their 
past  work  experience,  making  it  difficult  to  precisely  estimate  what  role  the  change  in  past-work- 
experience  characteristics  plays  in  any  difference  in  retention.  For  example,  a  smaller  fraction  of 
entrants  in  the  policy  cohort  group  are  enrolled  in  the  Civil  Service  Retirement  System  (CSRS) 
retirement  plan  relative  to  entrants  in  the  pre-policy  cohort  groups  (see  the  Tables  in  Appendix 
G).  Additionally,  among  other  things,  the  distributions  across  DoD  organizations  and  years-until- 
retirement-eligibility  groups  also  differ  between  the  policy  and  pre-policy  cohort  groups.  These 
fundamental  differences  could  play  a  role  in  retention  trends.  However,  given  the  results  from 
the  descriptive  analysis  (presented  in  Chapter  7)  and  from  Part  I  of  the  retention  analysis 
(presented  in  Chapter  8),  it  would  not  be  unreasonable  to  conjecture  that  the  shift  in  past-work- 
experience-characteristics  brought  about  by  the  growth  initiative  may  play  some  part  in  any 
differences  in  retention  that  may  exist. 

The  analysis  also  runs  these  comparisons  in  the  other  direction  to  check  the  robustness  of  the 
initial  findings.  Specifically,  the  same  overall  method  is  used  to  estimate  what  the  pre-policy 
retention  trends  would  have  been  if  these  pre-policy  cohorts  had  looked  more  like  the  policy 
cohorts.  To  do  this,  three  synthetic  cohorts  are  created:  First,  the  early  pre-policy  cohort  group  is 
weighted  to  look  more  like  the  policy  cohort  group  based  on  most  of  the  characteristics  listed  in 
Table  6.265  and  on  the  prior-career-profile  characteristics — this  synthetic  cohort  group  is 
hereafter  referred  to  as  “synthetic  policy  cohort  group  1.”  Then,  the  later  pre-policy  cohort  group 
is  weighted  to  look  like  the  policy  cohort  group — this  synthetic  cohort  group  is  hereafter  called 
“synthetic  cohort  group  2.”  Lastly,  an  additional  synthetic  policy  cohort  group  is  created, 
“synthetic  policy  cohort  group  3,”  by  weighting  the  combination  pre -policy  cohort  group  to  look 
like  the  policy  cohort  group.66  After  creating  synthetic  policy  cohort  groups  1-3,  the  analysis 
compares  retention  trends  between  the  unweighted  pre-policy  cohort  groups  and  their  respective 
synthetic  policy  cohort  groups.  Cox  PH  models  are  used  to  test  for  differences  in  retention.  The 
models  include  multivariate  controls  for  specific  cohort  year.  Table  9.3,  below,  outlines  the  thre 
comparisons. 


65  The  following  characteristics  are  not  used  in  the  creation  of  the  synthetic  cohort  groups:  Cohort  Year  and 
LN(Compensation).  Cohort  Year  is  not  included  because  it  is  impossible  to  weight  one  cohort  group  to  match  the 
cohort  year  distribution  of  another  cohort  group.  LN(Compensation)  is  not  included  because,  in  general,  nominal 
compensation  levels  tend  to  increase  over  time,  so  it  would  be  inaccurate  to  weight  one  cohort  group  to  match  the 
compensation  distribution  of  another  cohort  group.  For  the  creation  of  the  synthetic  cohorts,  the  Broad  GS  Pay  Plan 
group  and  the  NSPS  Pay  Plan  group  are  combined  into  one  group,  which  is  called  “Broad  GS  or  NSPS.” 

66  Appendix  G  explains  in  more  detail  how  these  synthetic  cohort  groups  are  created.  Appendix  G  also  shows  that 
the  weighting  schemes  used  to  create  the  synthetic  policy  cohort  groups  are  successful — the  variable  means  from  the 
synthetic  policy  cohort  groups  resemble  those  from  the  actual  policy  cohort  group. 
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Table  9.3.  To  Understand  What  Pre-Policy  Cohort  Retention  Trends  Would  Have  Been  had  the  Pre- 
Policy  Cohorts  Looked  More  Like  Policy  Cohorts,  the  Analysis  Runs  the  Following  Comparisons 


Use  Cox  PH  models  to... 


1 .  Compare  retention  trends  from  the  early  pre-policy  cohort  group  to  retention  trends  from  synthetic 
policy  cohort  group  1a 

2.  Compare  retention  trends  from  the  later  pre-policy  cohort  group  to  retention  trends  from  synthetic 
policy  cohort  group  2b 

3.  Compare  retention  trends  from  the  combination  pre-policy  cohort  group  to  retention  trends  from 
synthetic  policy  cohort  group  3C 


a  Synthetic  policy  cohort  group  1  is  the  early  pre-policy  cohort  group  weighted  to  look  like  the  policy  cohort 
group 

b  Synthetic  policy  cohort  group  2  is  the  later  pre-policy  cohort  group  weighted  to  look  like  the  policy  cohort 
group 

c  Synthetic  policy  cohort  group  3  is  the  combination  pre-policy  cohort  group  weighted  to  look  like  the  policy 
cohort  group 


One  major  advantage  of  the  synthetic-cohort  technique,  which  compares  weighted  and 
unweighted  versions  of  the  same  dataset,  is  that  it  effectively  controls  for  any  global  time- 
specific  confounding  factors,  such  as  the  strength  of  the  national  economy  and  labor  market. 

As  in  Chapter  8,  the  following  analyses  focus  on  substantive  transfers  out  and  separations; 
administrative  transfers  out  are  censored  in  all  cases  since  these  represent  bureaucratic  changes 
rather  than  actual  personnel  movement.  In  other  words,  the  term  “retention”  (in  regard  to 
retention  in  the  civilian  DAW)  refers  to  an  individual’s  hazard  of  substantively  transferring  out 
of  or  separating  from  the  civilian  DAW.  Individuals  with  a  lower  hazard  of  substantively 
transferring  out  of  or  separating  from  the  civilian  DAW  (i.e.,  those  who  are  less  likely  to 
substantively  transfer  out  or  separate  in  any  given  year)  are  said  to  have  greater  retention. 


If  Policy  Cohorts  Had  Looked  More  Like  Pre-Policy  Cohorts,  Retention 
Would  Have  Been  Lower 

The  first  Cox  PH  models  compare  the  retention  patterns  of  the  synthetic  pre-policy  cohort 
groups  to  the  unweighted  policy  cohort  group.  Table  9.4  displays  the  results.  All  three  tests 
estimate  a  hazard  ratio  greater  than  one,  meaning  that  the  synthetic  pre-policy  cohort  groups 
have  lower  retention  than  the  policy  cohort  group.  Specifically,  the  models  estimate  that 
individuals  in  the  synthetic  pre-policy  cohort  group  are  approximately  11-12  percent  more  likely, 
compared  to  those  in  the  policy  cohort  group,  to  substantively  transfer  out  of  or  separate  from  the 
civilian  DAW  in  any  given  year. 
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Table  9.4.  Individuals  in  the  Synthetic  Pre-Policy  Cohort  Groups  Have  a  Greater  Hazard  of 
Substantively  Transferring  Out  of  or  Separating  From  the  Civilian  DAW  Relative  to  Individuals  in 

the  Policy  Cohort  Group 


Cox  PH 

Synthetic  Early  Pre-Policy 
Cohort  Group3 

Synthetic  Later  Pre-Policy 
Cohort  Group15 

Synthetic  Combination 
Pre-Policy  Cohort  Group3 

N 

118896 

118896 

118896 

Estimation 

0.110 

0.102 

0.108 

Hazard  Ratio 

1.116 

1.107 

1.114 

Standard  Error 

0.018 

0.013 

0.015 

p- value 

<.0001 

<.0001 

<.0001 

aThe  policy  cohort  group  weighted  to  look  like  the  early  pre-policy  cohort  group 
bThe  policy  cohort  group  weighted  to  look  like  the  later  pre-policy  cohort  group 
cThe  policy  cohort  group  weighted  to  look  like  the  combination  pre-policy  cohort  group 


This  result  suggests  that  the  growth  initiative  may  have  produced  cohorts  that  actually  tend  to 
have  greater  civilian  DAW  retention. 


If  Pre-Policy  Cohorts  had  Looked  More  Like  Policy  Cohorts,  Overall 
Retention  Would  Have  Been  Higher 

Next,  the  analysis  uses  Cox  PH  models  to  test  for  differences  in  retention  trends  between  the 
pre-policy  cohort  groups  and  their  respective  synthetic  policy  cohort  groups.  The  results  are 
outlined  in  Table  9.5.  The  results  reveal  that — on  average — the  synthetic  policy  cohort  groups 
have  higher  rates  of  retention  relative  to  their  respective  pre-policy  cohort  groups.  Specifically, 
the  models  estimate  that  individuals  in  the  synthetic  policy  cohort  groups  are  approximately  four 
to  seven  percent  less  likely  to  substantively  transfer  out  of  or  separate  from  the  civilian  DAW  in 
any  given  year  compared  to  individuals  in  the  pre-policy  cohort  groups. 

Table  9.5.  Individuals  in  the  Synthetic  Policy  Cohort  Groups  Have  a  Lower  Hazard  of  Substantively 
Transferring  Out  of  or  Separating  From  the  Civilian  DAW  Relative  to  Individuals  in  the  Pre-Policy 

Cohort  Groups 


Cox  PH 

Synthetic  Policy  Cohort 
Group  1a 

Synthetic  Policy  Cohort 
Group  2b 

Synthetic  Policy  Cohort 
Group  3C 

N 

54448 

57286 

111734 

Estimation 

-0.045 

-0.071 

-0.066 

Hazard  Ratio 

0.956 

0.932 

0.936 

Standard  Error 

0.017 

0.014 

0.010 

p- value 

0.0065 

<.0001 

<.0001 

aThe  early  pre-policy  cohort  group  weighted  to  look  like  the  policy  cohort  group 
bThe  later  pre-policy  cohort  group  weighted  to  look  like  the  policy  cohort  group 
cThe  combination  pre-policy  cohort  group  weighted  to  look  like  the  policy  cohort  group 
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Similar  to  the  tests  presented  in  Table  9.4,  these  results  suggest  that  the  growth  initiative  may 
have  increased  overall  cohort-level  retention. 

The  Evidence  Suggests  that  the  Growth  Initiative  Increased  Cohort-Level 
Retention  Rates 

In  the  end,  the  synthetic  cohort  analysis  suggests  that  the  growth  initiative  may  have 
increased  overall  cohort-level  retention  rates.  Given  that  there  are  a  number  of  differences 
between  the  pre-policy  and  policy  cohort  groups,  it  is  difficult  to  say  what  role  the  change  in 
past-work-experience  characteristics  plays  in  this  difference  in  retention.  Nonetheless,  given  the 
results  presented  in  Chapters  7  and  8,  it  is  not  unreasonable  to  conjecture  that  the  shift  in  past- 
work-experience  characteristics  may  play  some  part:  The  descriptive  analysis  illustrates  that  the 
growth  initiative  was  fueled  by  outside  hires  with  no  prior  DoD  experience,  and  retention 
analysis  Part  I  shows  some  evidence  that  these  DoD  newcomers — in  general — tend  to  have  the 
greatest  propensity  to  stay  in  the  civilian  DAW.  Given  only  this  information,  one  would  predict  a 
positive  relationship  between  the  growth  initiative  and  retention,  and  that  is  what  this  synthetic- 
cohort  analysis  has  found.  At  the  same  time,  however,  there  are  likely  other  factors,  other  than 
past  work  experience,  that  contribute  to  this  difference  in  retention  trends. 
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Chapter  10.  Conclusions  and  Discussion 


The  goals  of  the  growth  initiative,  and  the  policies  that  helped  fuel  the  hiring  surge  (i.e.,  the 
DAWDF  and  the  Expedited  Hiring  Authority),  were  to  increase  the  size  of  the  workforce,  which 
some  had  considered  to  be  inadequate,  and  limit  the  consequences  of  the  forecasted  increase  in 
attrition.  Although  it  is  clear  that  the  growth  initiative  was  successful  in  increasing  the  size  of  the 
overall  DAW,  this  analysis  aimed  to  uncover  what  implications  this  strong  surge  in  hiring  might 
have  on  the  makeup  of  the  workforce,  in  terms  of  past-work-experience  characteristics,  and  on 
overall  rates  of  retention. 

This  study  utilizes  DoD  personnel  data  to  (1)  describe  who  is  joining  the  civilian  DAW,  in 
terms  of  past  work  experience,  and  illustrate  how  the  hiring  surge  has  changed  the  past-work- 
experience  characteristics  of  civilian  DAW  cohorts;  (2)  evaluate  how  prior  work  experience 
relates  to  retention  in  the  civilian  DAW;  and  (3)  estimate  how  the  growth  initiative  has 
influenced  overall  cohort-level  retention  rates.  In  the  end,  the  analyses  reveal  that  the  growth 
initiative  was  fueled  by  outside  hires  with  no  prior  DoD  experience  and  some  evidence  suggests 
that  these  DoD  newcomers — in  general — tend  to  have  the  highest  retention  in  the  civilian 
DAW.67  Additionally,  the  analyses  reveal  that  internal  hires  are  more  attached  to  the  DoD 
civilian  workforce  than  are  external  hires.  In  line  with  these  conclusions,  the  synthetic-cohort 
analysis  finds  that  the  hiring  surge  likely  produced  cohorts  with  greater  civilian  DAW  retention. 
These  findings,  and  their  policy  implications,  are  discussed  below. 

(1)  The  Growth  Initiative  was  Fueled  by  Outside  Hires  with  No  Prior  DoD 
Experience 

After  constructing  past-work-experience  descriptors  for  each  entrant  in  the  FY  2000-2014 
civilian  DAW  cohorts,  the  descriptive  analysis  examines  the  cohorts  to  understand  how  the  past- 
work-experience  characteristics  change  during  the  hiring  surge  years  (FYs  2008-2011).  Given 
that  part  of  the  growth  initiative  was  aimed  at  “converting”  “contractors”  into  actual  DoD 
personnel  (Gates,  2009a),  the  analysis  hypothesized  that  many  of  the  hires  during  the  growth 
initiative  years  would  come  from  positions  outside  of  the  DoD,  and  that  is  what  the  descriptive 
analysis  finds.  Even  more,  the  study  shows  that  more  than  half  of  the  outside  hires  who  were 
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One  exception:  Test  results  in  Table  8.1 1  show  some  evidence  suggesting  that  external  hires  with  the  Active  Duty 
+  Military  DAW  prior  career  profile  may  have  greater  retention  in  the  civilian  DAW  than  entrants  with  no  prior 
DoD  experience. 
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As  outlined  in  “Chapter  6.  Data  and  Definitions,”  individuals  are  marked  as  being  in  the  DoD  for  a  particular  FY 
if  they  appear  in  either  the  end-of-FY  ADMF  or  end-of-FY  DoD  civilian  personnel  file  for  that  particular  FY. 
Individuals  who  do  not  appear  in  one  of  these  files  are  considered  to  be  outside  of  the  DoD  for  that  FY.  Because 
defense  contractors  are  not  included  in  the  ADMFs  or  DoD  civilian  personnel  files,  they  are  marked  as  being  outside 
of  the  DoD. 
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brought  in  during  the  hiring  surge  did  not  have  any  prior  experience  in  the  DoD.  Ultimately,  the 
analysis  reveals  that  cohort  makeup,  in  terms  of  prior  work  experience,  shifted  considerably 
during  the  growth  initiative:  Hiring  surge  cohorts  have  a  significantly  greater  percentage  of 
outside  hires  with  no  prior  DoD  experience  compared  to  the  cohorts  before  the  hiring  surge. 

(2)  Some  Evidence  Suggests  that  These  DoD  Newcomers — in  General — 
Have  the  Greatest  Retention  in  the  Civilian  DAW 

Retention  analysis  Part  I  addresses  the  second  research  question.  Specifically,  the  analysis 
examines  the  relationship  between  prior  experience  in  the  DoD  and  civilian  DAW  retention 
outcomes.  In  the  end,  the  analysis  fails  to  support  the  original  hypothesis — that  entrants  with 
prior  DoD  experience,  particularly  those  with  prior  experience  in  the  DoD  civilian  workforce  or 
civilian  DAW,  would  have  greater  retention  in  the  civilian  DAW  than  entrants  with  no  prior 
DoD  experience.  In  fact,  some  results  suggest  the  opposite.  Specifically,  the  analysis  finds  some 
evidence  that  internal  hires  (entrants  who  join  directly  from  a  non-DAW  DoD  civilian  position) 
tend  to  have  lower  civilian  DAW  retention  than  external  hires  (entrants  who  come  directly  from 
a  military  position — DAW  or  otherwise — or  from  outside  of  the  DoD  entirely),  and  additional 
tests  find  that,  among  external  hires,  those  with  no  prior  DoD  experience — in  general — tend  to 
have  the  greatest  retention  in  the  civilian  DAW.69 

(3)  Internal  Hires  are  More  Attached  to  the  DoD  Civilian  Workforce  than  are 
External  Hires 

The  tests  that  examine  substantive  transfers  out  and  separations  individually,  as  well  as  tests 
on  retention  in  the  wider  DoD  civilian  workforce,  shed  some  light  on  the  potential  mechanisms 
behind  these  unexpected  findings.  In  particular,  the  tests  show  that  internal  hires  are  (1) 
significantly  more  likely,  than  external  hires,  to  substantively  transfer  out  of  the  civilian  DAW 
(into  the  wider  non-DAW  DoD  civilian  workforce)  and  (2)  significantly  less  likely  to  separate 
from  the  civilian  DAW  (leave  the  DoD  civilian  workforce  entirely)  in  any  given  year.  These 
results  suggest  that  internal  hires  are  much  more  comfortable,  relative  to  external  hires,  moving 
from  a  civilian  DAW  position  to  a  position  in  the  wider  DoD  civilian  workforce.  In  fact,  analyses 
on  retention  in  the  wider  DoD  civilian  workforce  show  that  internal  hires  have  greater  retention 
in  the  wider  DoD  civilian  workforce  than  do  external  hires. 


69 

One  exception:  Test  results  in  Table  8.1 1  show  some  evidence  suggesting  that  external  hires  with  the  Active  Duty 
+  Military  DAW  prior  career  profile  may  have  greater  retention  in  the  civilian  DAW  than  entrants  with  no  prior 
DoD  experience. 
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(4)  The  Growth  Initiative  Likely  Increased  Cohort-Level  Retention  Rates 

The  second  retention  analysis  uses  a  synthetic-cohort  analysis  technique  to  address  the  third 
and  final  research  question,  which  asks  how  the  hiring  surge  affected  overall  cohort-level 
retention  rates.  In  the  end,  the  analysis  suggests  that  the  growth  initiative  may  have  produced 
cohorts  with  greater  civilian  DAW  retention  (relative  to  what  would  have  happened  had  the  pre- 
policy  cohort  trends  continued).  This  outcome  is  counter  to  the  original  hypothesis  but  makes 
sense  given  the  findings  from  the  descriptive  analysis  and  from  retention  analysis  Part  I — that  the 
hiring  surge  was  mostly  fueled  by  outside  hires  with  no  prior  DoD  experience  and  that  some 
evidence  suggests  that  these  DoD  newcomers,  in  general,  tend  to  have  the  greatest  propensity  to 
stay  in  the  civilian  DAW. 

Policy  Implications 

The  findings  listed  above  have  important  policy  implications.  First,  at  the  strategic  workforce 
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planning  level,  the  findings  suggest  that  DAW  “rightsizing”  initiatives  do  more  than  influence 
workforce  size;  the  policies  also  influence  workforce  characteristics  and  outcomes,  such  as 
retention.  Policymakers  at  the  Office  of  the  Secretary  of  Defense  and  Undersecretary  of  Defense 
for  Acquisition,  Technology  and  Logistics  should  consider  these  factors  when  enacting  policy  to 
guide  workforce  growth  initiatives  or  downsizing  efforts.  Although  there  are  certainly  a  number 
of  factors  that  influence  workforce  retention,  the  findings  presented  in  this  study  suggest  that 
hiring  surges  that  bring  in  outside  hires  with  no  prior  DoD  experience  may  produce  cohorts  with 
greater  levels  of  retention.  On  the  other  hand,  a  hiring  surge  accomplished  by  increasing  the 
number  of  internal  hires  from  the  wider  DoD  civilian  workforce  may  produce  cohorts  with  lower 
retention  rates. 

These  findings  also  have  retention-policy  implications.  Specifically,  the  results  suggest  that 
policies  aimed  at  improving  civilian  DAW  retention  should  distinguish  between  internal  and 
external  hires  as  the  two  entrant  types  exhibit  different  civilian  DAW  loss-type  patterns.  The 
study  shows  that  internal  hires  are  more  likely  to  substantively  transfer  out  of  but  less  likely  to 
separate  from  the  civilian  DAW  in  any  given  year  compared  to  external  hires.  This  suggests  that 
internal  hires  may  identity  more  as  DoD  civilians  than  do  external  hires.  Therefore,  policies 
aimed  at  improving  civilian  DAW  retention  among  internal  hires  should  focus  on  reducing 
substantive  transfers  out.  For  example,  policies  targeted  at  internal  hires  could  focus  on  forging 
entrants’  identity  as  acquisition  professionals,  thereby  making  internal  hires  feel  more  attached  to 
the  acquisition  workforce  specifically.  On  the  other  hand,  policies  aimed  at  improving  civilian 
DAW  retention  among  external  hires  should  focus  on  reducing  separations.  For  example, 
policies  targeted  at  external  hires  could  emphasize  integrating  external  hires  into  the  DoD 


70  Gates  et  al.  (2008)  write,  “A  popular  term  used  in  discussing  the  management  of  government  agencies  is 
‘rightsizing’ — having  the  optimal  or  appropriate  number  of  employees  to  accomplish  the  required  tasks”  (p.  3).  This 
study  also  uses  the  term  “rightsizing.” 


72 


civilian  workforce  more  broadly.  Further  research  (perhaps  a  qualitative  investigation)  is  needed 
to  recommend  more-specific  policies  to  improve  retention  among  civilian  DAW  entrants.  At  a 
minimum,  however,  this  study  suggests  that  policies  aimed  at  improving  civilian  DAW  retention 
should  distinguish  between  internal  and  external  hires. 

Lastly,  the  results  can  also  help  inform  individual-level  hiring  decisions.  For  example,  the 
analysis  shows  some  evidence  suggesting  that  among  similar  entrants,  internal  hires  (i.e.,  those 
who  transferred  into  the  civilian  DAW  from  a  non-DAW  DoD  civilian  position)  tend  to  have 
shorter  tenure  as  civilian  defense  acquisition  professionals  than  external  hires  (i.e.,  those  who 
joined  from  a  military  position — DAW  or  otherwise — or  from  a  position  outside  of  the  DoD). 
Personnel  managers  should  consider  these  trends  when  making  hiring  decisions.  However, 
further  research  is  needed  to  understand  whether  this  relationship  holds  across  all  of  the 
acquisition  career  fields. 

Study  Limitations  and  Further  Research 

This  study  presents  a  detailed  examination  of  the  civilian  DAW,  the  recent  growth  initiative, 
and  how  past  work  experience  relates  to  retention  outcomes,  yielding  interesting  and  useful 
results.  However,  it  does  have  its  limitations,  presenting  opportunities  for  further  research. 

First,  one  of  the  study’s  limitations  is  that  it  uses  annual  end-of-FY  personnel  inventory  files 
to  (1)  define  each  civilian  DAW  cohort;  (2)  characterize  prior  work  experience;  and  (3)  measure 
retention.  Using  administrative  end-of-FY  data  is  an  effective  but  imperfect  way  to  define 
cohorts  and  characterize  past  work  experience — this  technique  misses  some  individuals  and 
miscategorizes  others.  In  particular,  individuals  who  are  not  in  a  civilian  DAW  position  at  the 
end  of  a  FY  (in  this  study,  at  the  end  of  FYs  2000  to  2014)  are  not  included  in  the  cohort  for  that 
FY.  Similarly,  in  terms  of  characterizing  prior  work  experience,  the  annual  files  do  not  capture 
the  prior  DoD  work  experience  for  individuals  who  are  not  in  a  particular  DoD  workforce  at  the 
end  of  the  FY.71  Retention  infonnation  is  also  derived  from  these  end-of-FY  inventory  files, 
making  the  retention  data  also  approximate.  Future  analyses  could  utilize  the  personnel 
transaction  data,  which  contain  precise  start  and  end  dates,  to  address  these  limitations  and 
produce  improved  estimates. 

Another  limitation  is  that  this  study  only  uses  DoD  data  to  characterize  prior  work 
experience.  Although  the  majority  of  entrants  analyzed  in  this  study  have  some  prior  DoD  work 
experience  (see  Table  6. 1),  little  is  known  about  the  work  history  for  those  individuals  who  do 
not  have  prior  experience  in  the  DoD.  These  individuals  could  have  a  range  of  different  work 
experiences  outside  of  the  DoD.  For  example,  among  other  possibilities,  these  individuals  could 
be  experienced  government  contractors  with  years  of  acquisition  experience,  federal  civil 
servants  who  have  never  worked  under  the  Defense  Department,  or  even  recent  college  graduates 
with  no  work  experience  at  all.  These  various  past  work  experiences  could  have  different  effects 


71  Although,  prior  work  experience  that  is  less  than  one  year  in  duration  may  not  be  too  influential. 
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on  retention  outcomes  in  the  civilian  DAW.  Further  research  that  expands  on  the  data  used  to 
characterize  prior  work  experience — potentially  including  federal  and  private-sector  personnel 
data  files — would  improve  the  understanding  of  how  prior  work  experience  relates  to  civilian 
DAW  retention  outcomes. 

Additionally,  this  study  uses  broad  categories  to  characterize  prior  work  experience: 
experience  in  the  active-duty  military  (non-DAW);  experience  in  the  wider  (non-DAW)  DoD 
civilian  workforce;  experience  in  the  military  DAW;  and  prior  experience  in  the  civilian  DAW. 
Although  the  analysis  distinguishes  between  acquisition  and  non-acquisition  experience,  further 
distinctions  between  different  types  of  jobs  within  these  broader  workforces  could  highlight 
more  specific  connections  between  prior  work  experience  and  retention. 

This  analysis  examines  the  civilian  DAW  entrants  as  a  whole.  However,  the  relationship 
between  prior  work  experience  and  retention  outcomes  could  vary  across  career  fields  within  the 
civilian  DAW.  Future  analyses  should  explore  how  certain  past  work  experiences  influence 
retention  in  the  top  acquisition  career  fields.  This  would  help  guide  hiring  decisions  within  those 
career  fields. 

This  study  uses  the  Cox  PH  model  to  test  for  differences  in  retention  trends.  This  model  is 
“robust”  in  that  it  makes  no  assumption  regarding  the  underlying  hazard  function  (Allison,  2010, 
pp.  125-126).  The  model  also  allows  for  the  incorporation  of  weights  and  controls.  However,  the 
Cox  PH  model  is  limited  in  that  it  assumes  a  constant  hazard  ratio — one  that  does  not  vary  with 
time  in  the  workforce  (see  appendix  B).  Further  analyses  should  use  alternative  methods  to 
explore  these  data.  This  would  add  additional,  perhaps  more  nuanced,  insights  into  the 
relationship  between  prior  work  experience  and  civilian  DAW  retention. 

Lastly,  while  this  study  focuses  on  retention,  future  analyses  could  explore  the  relationship 
between  prior  work  experience  and  other  workforce  outcomes,  such  as  performance  or  career 
progression.  Such  analyses  would  help  managers  make  more-informed  decisions  when  hiring 
new  personnel  and  recommending  strategic  workforce  plans. 
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Appendix  A.  Dropping  Observations  Does  Not  Significantly 
Change  Cohort  Characteristics 


In  order  to  examine  whether  dropping  observations  changes  cohort  characteristics,  the 
analysis  data  fde  is  divided  into  four  cohort  groups,  reflecting  the  groups  used  in  Parts  I  and  II  of 
the  retention  analysis  (see  Table  A.l).  Sections  A.l  through  A.4,  below,  show  that  dropping 
observations  due  to  missing  information  does  not  significantly  change  the  composition  of  the 
cohort  groups,  based  on  the  characteristics  listed  in  Table  6.2.  The  “Absolute  standard 
difference”  between  variable  means  before  and  after  dropping  observations  never  exceeds  0.06. 


Table  A.l.  Pre-Policy  and  Policy  Cohort  Groups  for  Retention  Analyses 

Cohort  Group  Name  FY  Cohorts  Included 

Early  Pre-policy  cohort  group  2001-2003 

Later  pre-policy  cohort  group  2004,  2005,  and  2007 

Combination  pre-policy  cohort  group  2001-2005  and  2007 
Policy  cohort  group  2008-2011 


A.l  Early  Pre-Policy  Cohort  Group 

Table  A.1.1.  Variable  Means  and  Standard  Deviations  Before  and  After  Dropping  Observations, 

Early  Pre-Policy  Cohort  Group 


After  Drops 

Before  Drops 

Standard 

Difference 

Parameter 

Mean 

SD 

Mean 

SD 

in  Means 

1 

AGE 

40.347 

1 1 .006 

40.311 

1 1 .022 

0.003 

2 

AGE:  Missing 

0 

0 

0 

0.015 

3 

LN(COMPENSATION) 

10.801 

0.325 

10.798 

0.331 

0.008 

4 

LN(COMPENSATION):  Missing 

0 

0 

0.007 

0.084 

5 

RACE:  Other 

0.23 

0.421 

0.228 

0.42 

0.004 

6 

RACE:  Unknown/Missing 

0 

0 

0.005 

0.071 

7 

RACE:  White 

0.77 

0.421 

0.767 

0.423 

0.008 

8 

EDUCATION:  BA 

0.454 

0.498 

0.453 

0.498 

0.003 

9 

EDUCATION:  Less  than  BA 

0.331 

0.471 

0.33 

0.47 

0.003 
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“  This  phrase  stems  from  Griffin  and  McCaffrey  (2015c)  (p.  28).  The  phrase  is  used  throughout  the  following 
section. 
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10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 

26 

27 

28 

29 

30 

31 

32 

33 

34 

35 

36 

37 

38 

39 

40 

41 

42 

43 

44 

45 

46 

47 

48 

49 

50 

51 


Parameter 

After  Drops 

Mean  SD 

Before  Drops 

Mean  SD 

Standard 
Difference 
in  Means 

EDUCATION:  Missing 

0 

0 

0.002 

0.046 

EDUCATION:  More  than  BA 

0.215 

0.411 

0.215 

0.411 

-0.002 

DOD  ORG:  Air  Force 

0.151 

0.358 

0.15 

0.357 

0.004 

DOD  ORG:  Army 

0.413 

0.492 

0.414 

0.493 

-0.003 

DOD  ORG:  Navy 

0.342 

0.475 

0.343 

0.475 

-0.001 

DOD  ORG:  Other 

0.094 

0.292 

0.093 

0.291 

0.002 

REGION:  East  North  Central 

0.105 

0.306 

0.104 

0.305 

0.003 

REGION:  East  South  Central 

0.084 

0.278 

0.084 

0.277 

0 

REGION:  Mid  Atlantic 

0.1 

0.3 

0.1 

0.3 

0.002 

REGION:  Mountain 

0.065 

0.247 

0.065 

0.247 

-0.002 

REGION:  New  England 

0.04 

0.196 

0.04 

0.196 

0.001 

REGION:  Other  or  Missing 

0.029 

0.169 

0.029 

0.168 

0.001 

REGION:  Pacific 

0.151 

0.358 

0.151 

0.358 

0 

REGION:  South  Atlantic 

0.339 

0.473 

0.341 

0.474 

-0.004 

REGION:  West  North  Central 

0.014 

0.116 

0.014 

0.116 

0.001 

REGION:  West  South  Central 

0.073 

0.26 

0.072 

0.259 

0.001 

GENDER:  Female 

0.36 

0.48 

0.36 

0.48 

0 

GENDER:  Male 

0.64 

0.48 

0.639 

0.48 

0.001 

GENDER:  Missing 

0 

0 

0.001 

0.023 

POSITION  TYPE:  Administrative 

0.4 

0.49 

0.396 

0.489 

0.006 

POSITION  TYPE:  Other 

0.014 

0.117 

0.015 

0.121 

-0.01 

POSITION  TYPE:  Professional 

0.536 

0.499 

0.537 

0.499 

-0.004 

POSITION  TYPE:  Technical 

0.051 

0.22 

0.051 

0.22 

0.001 

POSITION  TYPE:  Unknown/Missing 

0 

0 

0 

0.015 

SUPERVISOR:  All  Other  Positions 

0.919 

0.273 

0.919 

0.272 

-0.001 

SUPERVISOR:  Supvisor/Mngr/Ldr 

0.081 

0.273 

0.081 

0.272 

0.001 

ACQ  CAREER  FLD:  Business  (CE  or  FM) 

0.088 

0.284 

0.088 

0.283 

0.003 

ACQ  CAREER  FLD:  Contracting 

0.151 

0.358 

0.151 

0.358 

-0.001 

ACQ  CAREER  FLD:  Engineering 

0.289 

0.453 

0.29 

0.454 

-0.003 

ACQ  CAREER  FLD:  Facilities  Engineering 

0.028 

0.165 

0.029 

0.167 

-0.004 

ACQ  CAREER  FLD:  Information  Tech 

0.071 

0.257 

0.071 

0.256 

0.002 

ACQ  CAREER  FLD:  Life  Cycle  Logistics 

0.096 

0.295 

0.095 

0.294 

0.002 

ACQ  CAREER  FLD:  Other 

0.038 

0.192 

0.038 

0.191 

0.001 

ACQ  CAREER  FLD:  Prod,  Quality  &  Manuf 

0.044 

0.205 

0.044 

0.205 

0.001 

ACQ  CAREER  FLD:  Program  Management 

0.087 

0.281 

0.087 

0.281 

0 

ACQ  CAREER  FLD:  Test  and  Eval 

0.051 

0.219 

0.051 

0.22 

-0.001 

ACQ  CAREER  FLD:  Unknown/Missing 

0.057 

0.232 

0.057 

0.232 

0.001 

RETIREMENT  PLAN:  CSRS 

0.201 

0.401 

0.199 

0.399 

0.006 

RETIREMENT  PLAN:  FERS 

0.793 

0.405 

0.794 

0.404 

-0.002 

RETIREMENT  PLAN:  Other 

0.006 

0.075 

0.007 

0.085 

-0.021 

CAREER  LEVEL:  Entry-Level 

0.265 

0.441 

0.266 

0.442 

-0.003 

CAREER  LEVEL:  Mid-Level 

0.663 

0.473 

0.661 

0.474 

0.004 
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After  Drops 

Before  Drops 

Standard 

Difference 

Parameter 

Mean 

SD 

Mean 

SD 

in  Means 

52 

CAREER  LEVEL:  Sen/Exec 

0.064 

0.244 

0.064 

0.245 

-0.001 

53 

CAREER  LEVEL:  Uncategorized 

0.009 

0.095 

0.01 

0.098 

-0.006 

54 

COHORT  YEAR:  2001 

0.308 

0.462 

0.305 

0.46 

0.006 

55 

COHORT  YEAR:  2002 

0.376 

0.484 

0.372 

0.483 

0.006 

56 

COHORT  YEAR:  2003 

0.317 

0.465 

0.322 

0.467 

-0.012 

57 

YRS  TO  RET  ELIG:  <5  Yrs  to  Go 

0.082 

0.274 

0.081 

0.273 

0.002 

58 

YRS  TO  RET  ELIG:  10  to  19  Yrs  to  Go 

0.337 

0.473 

0.337 

0.473 

-0.001 

59 

YRS  TO  RET  ELIG:  20+  YRs  to  Go 

0.392 

0.488 

0.393 

0.489 

-0.003 

60 

YRS  TO  RET  ELIG:  5-9  Yrs  to  Go 

0.145 

0.352 

0.144 

0.351 

0.003 

61 

YRS  TO  RET  ELIG:  Ret  Eligible 

0.045 

0.207 

0.044 

0.206 

0.002 

62 

YRS  TO  RET  ELIG:  Unknown/Missing 

0 

0 

0 

0.015 

63 

PAY  PLAN:  Broad  GS 

0.802 

0.398 

0.799 

0.4 

0.007 

64 

PAY  PLAN:  Demo 

0.19 

0.392 

0.192 

0.394 

-0.006 

65 

PAY  PLAN:  Other 

0.008 

0.09 

0.009 

0.093 

-0.007 

66 

HCAP:  Handicapped 

0.087 

0.282 

0.09 

0.285 

-0.008 

67 

HCAP:  Not  Handicapped 

0.913 

0.282 

0.91 

0.286 

0.01 

68 

HCAP:  Unknown/Missing 

0 

0 

0.001 

0.024 

Figure  A.1.1.  Early  Pre-Policy  Cohort  Group:  Absolute  Standard  Difference  in  Variable  Means  is 

Never  Greater  than  0.025 
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A.2  Later  Pre-Policy  Cohort  Group 


Table  A.2.1.  Variable  Means  and  Standard  Deviations  Before  and  After  Dropping  Observations, 

Later  Pre-Policy  Cohort  Group 


Parameter 

After  Drops 

Mean  SD 

Before  Drops 

Mean  SD 

Standard 
Difference 
in  Means 

i 

AGE 

39.257 

1 1 .492 

39.297 

11.573 

-0.004 

2 

LN(COMPENSATION) 

10.834 

0.349 

10.827 

0.359 

0.019 

3 

LN(COMPENSATION):  Missing 

0 

0 

0.002 

0.043 

4 

RACE:  Other 

0.223 

0.416 

0.218 

0.413 

0.011 

5 

RACE:  Unknown/Missing 

0 

0 

0.017 

0.13 

6 

RACE:  White 

0.777 

0.416 

0.765 

0.424 

0.03 

7 

EDUCATION:  BA 

0.471 

0.499 

0.469 

0.499 

0.004 

8 

EDUCATION:  Less  than  BA 

0.289 

0.453 

0.288 

0.453 

0.003 

9 

EDUCATION:  Missing 

0 

0 

0.005 

0.069 

10 

EDUCATION:  More  than  BA 

0.24 

0.427 

0.239 

0.426 

0.003 

11 

DOD  ORG:  Air  Force 

0.151 

0.358 

0.15 

0.357 

0.003 

12 

DOD  ORG:  Army 

0.449 

0.497 

0.443 

0.497 

0.013 

13 

DOD  ORG:  Navy 

0.254 

0.435 

0.25 

0.433 

0.009 

14 

DOD  ORG:  Other 

0.147 

0.354 

0.158 

0.365 

-0.032 

15 

REGION:  East  North  Central 

0.126 

0.332 

0.124 

0.329 

0.007 

16 

REGION:  East  South  Central 

0.088 

0.283 

0.086 

0.281 

0.005 

17 

REGION:  Mid  Atlantic 

0.117 

0.321 

0.116 

0.32 

0.002 

18 

REGION:  Mountain 

0.066 

0.249 

0.069 

0.253 

-0.01 

19 

REGION:  New  England 

0.034 

0.182 

0.035 

0.183 

-0.002 

20 

REGION:  Other  or  Missing 

0.025 

0.155 

0.024 

0.154 

0.002 

21 

REGION:  Pacific 

0.116 

0.32 

0.117 

0.322 

-0.006 

22 

REGION:  South  Atlantic 

0.349 

0.477 

0.348 

0.476 

0.002 

23 

REGION:  West  North  Central 

0.016 

0.126 

0.016 

0.127 

-0.002 

24 

REGION:  West  South  Central 

0.064 

0.245 

0.065 

0.246 

-0.002 

25 

GENDER:  Female 

0.354 

0.478 

0.355 

0.479 

-0.002 

26 

GENDER:  Male 

0.646 

0.478 

0.645 

0.479 

0.002 

27 

GENDER:  Missing 

0 

0 

0 

0.012 

28 

POSITION  TYPE:  Administrative 

0.39 

0.488 

0.387 

0.487 

0.006 

29 

POSITION  TYPE:  Other 

0.011 

0.106 

0.014 

0.119 

-0.028 

30 

POSITION  TYPE:  Professional 

0.543 

0.498 

0.543 

0.498 

0 

31 

POSITION  TYPE:  Technical 

0.056 

0.23 

0.055 

0.229 

0.001 

32 

POSITION  TYPE:  Unknown/Missing 

0 

0 

0 

0.008 

33 

SUPERVISOR:  All  Other  Positions 

0.918 

0.274 

0.919 

0.272 

-0.004 

34 

SUPERVISOR:  Supvisor/Mngr/Ldr 

0.082 

0.274 

0.081 

0.272 

0.004 

35 

ACQ  CAREER  FLD:  Business  (CE  or  FM) 

0.075 

0.263 

0.074 

0.261 

0.004 

36 

ACQ  CAREER  FLD:  Contracting 

0.168 

0.374 

0.171 

0.377 

-0.009 
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37 

38 

39 

40 

41 

42 

43 

44 

45 

46 

47 

48 

49 

50 

51 

52 

53 

54 

55 

56 

57 

58 

59 

60 

61 

62 

63 

64 

65 

66 

67 


Parameter 

After  Drops 

Mean  SD 

Before  Drops 

Mean  SD 

Standard 
Difference 
in  Means 

ACQ  CAREER  FLD:  Engineering 

0.216 

0.412 

0.214 

0.41 

0.006 

ACQ  CAREER  FLD:  Facilities  Engineering 

0.074 

0.262 

0.073 

0.26 

0.004 

ACQ  CAREER  FLD:  Information  Tech 

0.04 

0.195 

0.039 

0.194 

0.003 

ACQ  CAREER  FLD:  Life  Cycle  Logistics 

0.127 

0.333 

0.125 

0.33 

0.008 

ACQ  CAREER  FLD:  Other 

0.03 

0.171 

0.031 

0.173 

-0.003 

ACQ  CAREER  FLD:  Prod,  Quality  &  Manuf 

0.073 

0.26 

0.074 

0.262 

-0.006 

ACQ  CAREER  FLD:  Program  Management 

0.074 

0.261 

0.073 

0.26 

0.004 

ACQ  CAREER  FLD:  Test  and  Eval 

0.043 

0.203 

0.042 

0.201 

0.003 

ACQ  CAREER  FLD:  Unknown/Missing 

0.08 

0.272 

0.084 

0.278 

-0.014 

RETIREMENT  PLAN:  CSRS 

0.109 

0.312 

0.108 

0.31 

0.005 

RETIREMENT  PLAN:  FERS 

0.882 

0.322 

0.879 

0.327 

0.011 

RETIREMENT  PLAN:  Other 

0.008 

0.091 

0.014 

0.116 

-0.059 

CAREER  LEVEL:  Entry-Level 

0.312 

0.463 

0.315 

0.465 

-0.007 

CAREER  LEVEL:  Mid-Level 

0.604 

0.489 

0.6 

0.49 

0.008 

CAREER  LEVEL:  Sen/Exec 

0.074 

0.261 

0.073 

0.261 

0.001 

CAREER  LEVEL:  Uncategorized 

0.01 

0.101 

0.011 

0.106 

-0.009 

COHORT  YEAR:  2004 

0.359 

0.48 

0.362 

0.481 

-0.007 

COHORT  YEAR:  2005 

0.348 

0.476 

0.345 

0.475 

0.006 

COHORT  YEAR:  2007 

0.294 

0.455 

0.293 

0.455 

0.001 

YRS  TO  RET  ELIG:  <5  Yrs  to  Go 

0.075 

0.263 

0.075 

0.263 

-0.001 

YRS  TO  RET  ELIG:  10  to  19  Yrs  to  Go 

0.312 

0.463 

0.311 

0.463 

0.002 

YRS  TO  RET  ELIG:  20+  YRs  to  Go 

0.455 

0.498 

0.455 

0.498 

0 

YRS  TO  RET  ELIG:  5-9  Yrs  to  Go 

0.118 

0.323 

0.118 

0.322 

0.002 

YRS  TO  RET  ELIG:  Ret  Eligible 

0.04 

0.195 

0.041 

0.199 

-0.009 

PAY  PLAN:  Broad  GS 

0.778 

0.416 

0.777 

0.416 

0.001 

PAY  PLAN:  Demo 

0.169 

0.375 

0.167 

0.373 

0.004 

PAY  PLAN:  NSPS 

0.044 

0.204 

0.044 

0.206 

-0.003 

PAY  PLAN:  Other 

0.01 

0.098 

0.011 

0.105 

-0.015 

HCAP:  Handicapped 

0.078 

0.269 

0.077 

0.267 

0.004 

HCAP:  Not  Handicapped 

0.922 

0.269 

0.922 

0.268 

-0.003 

HCAP:  Unknown/Missing 

0 

0 

0 

0.019 
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Absolute  standard  difference 


Figure  A.2.1.  Later  Pre-Policy  Cohort  Group:  Absolute  Standard  Difference  in  Variable  Means  is 

Never  Greater  than  0.06 
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A.3  Combination  Pre-Policy  Cohort  Group 


Table  A.3.1.  Variable  Means  and  Standard  Deviations  Before  and  After  Dropping  Observations, 

Combination  Pre-Policy  Cohort  Group 


Parameter 

After  Drops 

Mean  SD 

Before  Drops 

Mean  SD 

Standard 
Difference 
in  Means 

1 

AGE 

39.788 

11.271 

39.788 

11.321 

0 

2 

AGE:  Missing 

0 

0 

0 

0.01 

3 

LN(COMPENSATION) 

10.818 

0.338 

10.813 

0.346 

0.014 

4 

LN(COMPENSATION):  Missing 

0 

0 

0.004 

0.067 

5 

RACE:  Other 

0.226 

0.418 

0.223 

0.416 

0.008 

6 

RACE:  Unknown/Missing 

0 

0 

0.011 

0.106 

7 

RACE:  White 

0.774 

0.418 

0.766 

0.424 

0.019 

8 

EDUCATION:  BA 

0.463 

0.499 

0.461 

0.498 

0.003 

9 

EDUCATION:  Less  than  BA 

0.31 

0.462 

0.308 

0.462 

0.003 

10 

EDUCATION:  Missing 

0 

0 

0.003 

0.059 

11 

EDUCATION:  More  than  BA 

0.228 

0.419 

0.227 

0.419 

0.001 

12 

DOD  ORG:  Air  Force 

0.151 

0.358 

0.15 

0.357 

0.004 

13 

DOD  ORG:  Army 

0.431 

0.495 

0.429 

0.495 

0.005 

14 

DOD  ORG:  Navy 

0.297 

0.457 

0.295 

0.456 

0.005 

15 

DOD  ORG:  Other 

0.121 

0.326 

0.127 

0.333 

-0.018 

16 

REGION:  East  North  Central 

0.116 

0.32 

0.114 

0.318 

0.005 

17 

REGION:  East  South  Central 

0.086 

0.28 

0.085 

0.279 

0.003 

18 

REGION:  Mid  Atlantic 

0.109 

0.311 

0.108 

0.311 

0.002 

19 

REGION:  Mountain 

0.066 

0.248 

0.067 

0.25 

-0.006 

20 

REGION:  New  England 

0.037 

0.189 

0.037 

0.189 

-0.001 

21 

REGION:  Other  or  Missing 

0.027 

0.162 

0.027 

0.161 

0.002 

22 

REGION:  Pacific 

0.133 

0.339 

0.134 

0.34 

-0.002 

23 

REGION:  South  Atlantic 

0.344 

0.475 

0.345 

0.475 

-0.001 

24 

REGION:  West  North  Central 

0.015 

0.121 

0.015 

0.122 

-0.001 

25 

REGION:  West  South  Central 

0.068 

0.252 

0.068 

0.252 

0 

26 

GENDER:  Female 

0.357 

0.479 

0.358 

0.479 

-0.001 

27 

GENDER:  Male 

0.643 

0.479 

0.642 

0.479 

0.002 

28 

GENDER:  Missing 

0 

0 

0 

0.018 

29 

POSITION  TYPE:  Administrative 

0.395 

0.489 

0.392 

0.488 

0.006 

30 

POSITION  TYPE:  Other 

0.013 

0.111 

0.015 

0.12 

-0.018 

31 

POSITION  TYPE:  Professional 

0.539 

0.498 

0.54 

0.498 

-0.002 

32 

POSITION  TYPE:  Technical 

0.054 

0.225 

0.053 

0.225 

0.001 

33 

POSITION  TYPE:  Unknown/Missing 

0 

0 

0 

0.012 

34 

SUPERVISOR:  All  Other  Positions 

0.919 

0.273 

0.919 

0.272 

-0.003 

35 

SUPERVISOR:  Supvisor/Mngr/Ldr 

0.081 

0.273 

0.081 

0.272 

0.003 

36 

ACQ  CAREER  FLD:  Business  (CE  or  FM) 

0.081 

0.273 

0.08 

0.272 

0.004 
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37 

38 

39 

40 

41 

42 

43 

44 

45 

46 

47 

48 

49 

50 

51 

52 

53 

54 

55 

56 

57 

58 

59 

60 

61 

62 

63 

64 

65 

66 

67 

68 

69 

70 

71 

72 


Parameter 

After  Drops 

Mean  SD 

Before  Drops 

Mean  SD 

Standard 
Difference 
in  Means 

ACQ  CAREER  FLD:  Contracting 

0.16 

0.366 

0.162 

0.368 

-0.005 

ACQ  CAREER  FLD:  Engineering 

0.251 

0.434 

0.251 

0.433 

0.002 

ACQ  CAREER  FLD:  Facilities  Engineering 

0.052 

0.221 

0.052 

0.221 

0 

ACQ  CAREER  FLD:  Information  Tech 

0.055 

0.228 

0.054 

0.227 

0.003 

ACQ  CAREER  FLD:  Life  Cycle  Logistics 

0.112 

0.315 

0.11 

0.313 

0.005 

ACQ  CAREER  FLD:  Other 

0.034 

0.182 

0.034 

0.182 

-0.001 

ACQ  CAREER  FLD:  Prod,  Quality  &  Manuf 

0.059 

0.235 

0.06 

0.237 

-0.003 

ACQ  CAREER  FLD:  Program  Management 

0.08 

0.271 

0.08 

0.271 

0.002 

ACQ  CAREER  FLD:  Test  and  Eval 

0.047 

0.211 

0.046 

0.21 

0.001 

ACQ  CAREER  FLD:  Unknown/Missing 

0.069 

0.254 

0.071 

0.257 

-0.008 

RETIREMENT  PLAN:  CSRS 

0.154 

0.361 

0.152 

0.359 

0.006 

RETIREMENT  PLAN:  FERS 

0.839 

0.368 

0.838 

0.369 

0.003 

RETIREMENT  PLAN:  Other 

0.007 

0.083 

0.011 

0.102 

-0.042 

CAREER  LEVEL:  Entry-Level 

0.289 

0.453 

0.291 

0.454 

-0.005 

CAREER  LEVEL:  Mid-Level 

0.633 

0.482 

0.629 

0.483 

0.007 

CAREER  LEVEL:  Sen/Exec 

0.069 

0.253 

0.069 

0.253 

0 

CAREER  LEVEL:  Uncategorized 

0.01 

0.098 

0.011 

0.102 

-0.008 

COHORT  YEAR:  2001 

0.15 

0.357 

0.148 

0.355 

0.006 

COHORT  YEAR:  2002 

0.183 

0.387 

0.18 

0.384 

0.007 

COHORT  YEAR:  2003 

0.154 

0.361 

0.156 

0.363 

-0.005 

COHORT  YEAR:  2004 

0.184 

0.387 

0.187 

0.39 

-0.007 

COHORT  YEAR:  2005 

0.178 

0.383 

0.178 

0.382 

0.001 

COHORT  YEAR:  2007 

0.15 

0.358 

0.151 

0.358 

-0.002 

YRS  TO  RET  ELIG:  <5  Yrs  to  Go 

0.078 

0.268 

0.078 

0.268 

0.001 

YRS  TO  RET  ELIG:  10  to  19  Yrs  to  Go 

0.324 

0.468 

0.324 

0.468 

0.001 

YRS  TO  RET  ELIG:  20+  YRs  to  Go 

0.424 

0.494 

0.425 

0.494 

-0.002 

YRS  TO  RET  ELIG:  5-9  Yrs  to  Go 

0.131 

0.338 

0.13 

0.336 

0.003 

YRS  TO  RET  ELIG:  Ret  Eligible 

0.042 

0.201 

0.043 

0.203 

-0.003 

YRS  TO  RET  ELIG:  Unknown/Missing 

0 

0 

0 

0.01 

PAY  PLAN:  Broad  GS 

0.79 

0.408 

0.788 

0.409 

0.004 

PAY  PLAN:  Demo 

0.179 

0.383 

0.179 

0.383 

0 

PAY  PLAN:  NSPS 

0.022 

0.148 

0.023 

0.15 

-0.003 

PAY  PLAN:  Other 

0.009 

0.094 

0.01 

0.1 

-0.011 

HCAP:  Handicapped 

0.083 

0.275 

0.083 

0.276 

-0.002 

HCAP:  Not  Handicapped 

0.917 

0.275 

0.916 

0.277 

0.003 

HCAP:  Unknown/Missing 

0 

0 

0 

0.022 
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Absolute  standard  difference 


Figure  A.3.1.  Combination  Pre-Policy  Cohort  Group:  Absolute  Standard  Difference  in  Variable 

Means  is  Never  Greater  than  0.045 
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A.4  Policy  Cohort  Group 


Table  A.4.1.  Variable  Means  and  Standard  Deviations  Before  and  After  Dropping  Observations, 

Policy  Cohort  Group 


Parameter 

After  Drops 

Mean  SD 

Before  Drops 

Mean  SD 

Standard 
Difference 
in  Means 

i 

AGE 

39.372 

11.638 

39.385 

11.714 

-0.001 

2 

LN(COMPENSATION) 

10.938 

0.369 

10.93 

0.382 

0.02 

3 

LN(COMPENSATION):  Missing 

0 

0 

0 

0.008 

4 

RACE:  Other 

0.21 

0.407 

0.205 

0.403 

0.013 

5 

RACE:  Unknown/Missing 

0 

0 

0.025 

0.156 

6 

RACE:  White 

0.79 

0.407 

0.77 

0.421 

0.049 

7 

EDUCATION:  BA 

0.477 

0.499 

0.473 

0.499 

0.007 

8 

EDUCATION:  Less  than  BA 

0.241 

0.427 

0.245 

0.43 

-0.01 

9 

EDUCATION:  Missing 

0 

0 

0.002 

0.043 

10 

EDUCATION:  More  than  BA 

0.283 

0.45 

0.28 

0.449 

0.006 

11 

DOD  ORG:  Air  Force 

0.191 

0.393 

0.19 

0.392 

0.002 

12 

DOD  ORG:  Army 

0.323 

0.468 

0.325 

0.468 

-0.003 

13 

DOD  ORG:  Navy 

0.332 

0.471 

0.328 

0.47 

0.007 

14 

DOD  ORG:  Other 

0.154 

0.361 

0.157 

0.364 

-0.008 

15 

REGION:  East  North  Central 

0.135 

0.342 

0.133 

0.34 

0.006 

16 

REGION:  East  South  Central 

0.092 

0.289 

0.091 

0.287 

0.004 

17 

REGION:  Mid  Atlantic 

0.086 

0.281 

0.087 

0.281 

-0.001 

18 

REGION:  Mountain 

0.053 

0.224 

0.055 

0.228 

-0.009 

19 

REGION:  New  England 

0.035 

0.185 

0.035 

0.184 

0.002 

20 

REGION:  Other  or  Missing 

0.023 

0.149 

0.023 

0.15 

-0.001 

21 

REGION:  Pacific 

0.127 

0.333 

0.129 

0.335 

-0.006 

22 

REGION:  South  Atlantic 

0.375 

0.484 

0.372 

0.483 

0.006 

23 

REGION:  West  North  Central 

0.012 

0.107 

0.012 

0.108 

-0.001 

24 

REGION:  West  South  Central 

0.061 

0.24 

0.064 

0.244 

-0.01 

25 

GENDER:  Female 

0.325 

0.468 

0.328 

0.469 

-0.005 

26 

GENDER:  Male 

0.675 

0.468 

0.672 

0.469 

0.005 

27 

GENDER:  Missing 

0 

0 

0 

0.004 

28 

POSITION  TYPE:  Administrative 

0.405 

0.491 

0.403 

0.49 

0.005 

29 

POSITION  TYPE:  Other 

0.009 

0.093 

0.012 

0.111 

-0.039 

30 

POSITION  TYPE:  Professional 

0.535 

0.499 

0.534 

0.499 

0.003 

31 

POSITION  TYPE:  Technical 

0.051 

0.219 

0.051 

0.219 

-0.001 

32 

POSITION  TYPE:  Unknown/Missing 

0 

0 

0 

0.019 

33 

SUPERVISOR:  All  Other  Positions 

0.932 

0.252 

0.933 

0.251 

-0.003 

34 

SUPERVISOR:  Supvisor/Mngr/Ldr 

0.068 

0.252 

0.067 

0.251 

0.003 

35 

ACQ  CAREER  FLD:  Business  (CE  or  FM) 

0.075 

0.264 

0.075 

0.264 

0.001 

36 

ACQ  CAREER  FLD:  Contracting 

0.199 

0.399 

0.202 

0.402 

-0.008 
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37 

38 

39 

40 

41 

42 

43 

44 

45 

46 

47 

48 

49 

50 

51 

52 

53 

54 

55 

56 

57 

58 

59 

60 

61 

62 

63 

64 

65 

66 

67 

68 


Parameter 

After  Drops 

Mean  SD 

Before  Drops 

Mean  SD 

Standard 
Difference 
in  Means 

ACQ  CAREER  FLD:  Engineering 

0.213 

0.409 

0.211 

0.408 

0.003 

ACQ  CAREER  FLD:  Facilities  Engineering 

0.063 

0.242 

0.062 

0.242 

0.002 

ACQ  CAREER  FLD:  Information  Tech 

0.053 

0.224 

0.053 

0.224 

0 

ACQ  CAREER  FLD:  Life  Cycle  Logistics 

0.141 

0.348 

0.14 

0.347 

0.003 

ACQ  CAREER  FLD:  Other 

0.026 

0.159 

0.026 

0.159 

-0.001 

ACQ  CAREER  FLD:  Prod,  Quality  &  Manuf 

0.062 

0.242 

0.064 

0.244 

-0.005 

ACQ  CAREER  FLD:  Program  Management 

0.092 

0.29 

0.091 

0.288 

0.005 

ACQ  CAREER  FLD:  Test  and  Eval 

0.039 

0.195 

0.04 

0.195 

-0.001 

ACQ  CAREER  FLD:  Unknown/Missing 

0.036 

0.186 

0.036 

0.186 

0.001 

RETIREMENT  PLAN:  CSRS 

0.035 

0.184 

0.035 

0.185 

-0.003 

RETIREMENT  PLAN:  FERS 

0.955 

0.207 

0.951 

0.216 

0.022 

RETIREMENT  PLAN:  Other 

0.01 

0.099 

0.014 

0.117 

-0.04 

CAREER  LEVEL:  Entry-Level 

0.295 

0.456 

0.297 

0.457 

-0.004 

CAREER  LEVEL:  Mid-Level 

0.597 

0.49 

0.596 

0.491 

0.003 

CAREER  LEVEL:  Sen/Exec 

0.099 

0.299 

0.098 

0.298 

0.003 

CAREER  LEVEL:  Uncategorized 

0.008 

0.09 

0.008 

0.092 

-0.004 

COHORT  YEAR:  2008 

0.199 

0.399 

0.202 

0.401 

-0.007 

COHORT  YEAR:  2009 

0.281 

0.449 

0.281 

0.449 

0.001 

COHORT  YEAR:  2010 

0.312 

0.463 

0.309 

0.462 

0.005 

COHORT  YEAR:  201 1 

0.208 

0.406 

0.208 

0.406 

0 

YRS  TO  RET  ELIG:  <5  Yrs  to  Go 

0.068 

0.252 

0.069 

0.253 

-0.003 

YRS  TO  RET  ELIG:  10  to  19  Yrs  to  Go 

0.302 

0.459 

0.299 

0.458 

0.006 

YRS  TO  RET  ELIG:  20+  YRs  to  Go 

0.487 

0.5 

0.488 

0.5 

-0.001 

YRS  TO  RET  ELIG:  5-9  Yrs  to  Go 

0.111 

0.315 

0.112 

0.315 

-0.001 

YRS  TO  RET  ELIG:  Ret  Eligible 

0.031 

0.173 

0.032 

0.176 

-0.007 

PAY  PLAN:  Broad  GS 

0.632 

0.482 

0.632 

0.482 

-0.002 

PAY  PLAN:  Demo 

0.139 

0.346 

0.138 

0.344 

0.003 

PAY  PLAN:  NSPS 

0.216 

0.411 

0.216 

0.411 

0 

PAY  PLAN:  Other 

0.014 

0.117 

0.014 

0.119 

-0.004 

HCAP:  Handicapped 

0.082 

0.275 

0.082 

0.274 

0.001 

HCAP:  Not  Handicapped 

0.918 

0.275 

0.918 

0.275 

0 

HCAP:  Unknown/Missing 

0 

0 

0 

0.018 
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Absolute  standard  difference 


Figure  A.4.1.  Policy  Cohort  Group:  Absolute  Standard  Difference  in  Variable  Means  is  Never 

Greater  than  0.05 
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Appendix  B.  The  Cox  Proportional  Hazard  Model  and  Methods  for 
Controlling  for  Confounding  Factors 


This  appendix  explains,  in  more  detail,  (1)  the  Cox  Proportional  Hazard  (Cox  PH)  model, 
which  is  used  to  test  for  differences  in  retention,  and  (2)  the  methods  used  to  control  for 
confounding  factors. 

In  general,  the  data  used  for  the  retention  analyses  consist  of  four  parts  for  each  civilian 
DAW  entrant  included  in  the  analyses:  (1)  a  past-work-experience  category  descriptor,  (2)  a  loss- 
type  indicator,  (3)  a  tenure  descriptor,  and  (4)  other  observable  characteristics  (used  as 
control/balance  factors)  (Rich  et  ah,  2010,  pp.  2-3;  Allison,  2010,  p.  137).  The  loss-type  indicator 
indicates  whether  an  individual  experienced  a  loss  from  the  workforce,  and — in  the  case  of  the 
civilian  DAW — the  loss-type  indicator  specifies  which  particular  loss  type  an  individual 
experienced  (if  any)  (Rich  et  ah,  2010;  Allison,  2010).  Analyses  on  civilian  DAW  retention 
focus  on  substantive  transfers  out  and  separations;  administrative  transfers  out  are  censored. 
Because  this  study  uses  annual  data  files,  the  tenure  data  are  recorded  in  integer  years  (one  year, 
two  years,  three  years,  etc.).  The  past-work-experience  category  descriptor  allows  the  analyses  to 
test  for  differences  in  retention  trends  between  entrants  with  different  past  work  experiences 
(Rich  et  ah,  2010;  Allison,  2010). 

The  Cox  Proportional  Hazard  Model74 

The  Cox  PH  model  is  based  on  “the  hazard  function,  h(t)”  (Guo,  Hall-Partyka  and  Gates, 
2014,  p.6).  The  hazard  function,  for  the  purposes  of  this  analysis,  “describes  ....  the  probability 
that  an  individual”  is  marked  as  experiencing  a  loss  from  the  workforce  at  the  end  of  a  given 
tenure  year  (t),  “given  that”  he  or  she  did  not  experience  a  loss  in  previous  years  in  the 
workforce  (Guo,  Hall-Partyka  and  Gates,  2014,  p.6).  See  Equation  B.l. 


73 

All  analyses  on  civilian  DAW  retention  censor  administrative  transfers  out.  Analyses  that  specifically  examine 
substantive  transfers  out  of  the  civilian  DAW  censor  administrative  transfers  out  and  separations.  Analyses  that 
specifically  examine  separations  from  the  civilian  DAW  censor  administrative  transfers  out  and  substantive  transfers 
out. 

74 

The  Cox  Proportional  Hazard  model  is  named  after  David  Cox,  who  invented  this  hazard  model  estimation 
technique  (Cox,  1972;  Allison,  2010). 

Allison  (2010)  and  Guo,  Hall-Partyka  and  Gates  (2014)  provide  explanations  of  the  Cox  Proportional  Hazard  model. 
The  description  presented  in  this  study  is  based  on  the  explanations  from  Allison  (2010)  and  Guo,  Hall-Partyka  and 
Gates  (2014). 
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Equation  B.  1.  The  hazard  function 75 


P  ( individual  experiences  loss  from  workforce  at  end  of  year  t  in  workforce ) 

P  ( individual  still  in  workforce  at  end  of  year  t  in  workforce ) 

The  hazard  model  used  in  the  Cox  PH  model  is  shown  in  Equation  B.2,  below  (Guo,  Hall- 
Partyka  and  Gates,  2014,  p.6). 

Equation  B.  2.  The  hazard  model  used  in  the  Cox  PH  model76 

hk(t)  =  S(t)exp(/?Zfc) 

hk(t )  “is  the  hazard  of  [experiencing  a  loss]  for  individual  [A]”  (Guo,  Hall-Partyka  and 
Gates,  2014,  p.6)  “at  time  f  (Allison,  2010,  p.  127).  5(t)  represents  “the  baseline  hazard 
function”  (i.e.,  the  underlying  hazard  function,  if  all  covariates  were  zero)  (Guo,  Hall-Partyka 
and  Gates,  2014,  p.  6;  Allison,  2010,  p.  127).  /?  signifies  “a  vector  of  [estimated  covariate] 
coefficients,”  and  Zk  represents  “the  vector  of’  covariates  (i.e.,  observable  characteristics) 
included  in  the  analysis  for  individual  k  (Guo,  Hall-Partyka  and  Gates,  2014,  p.  6;  Allison, 

2010).  The  observable  characteristics  used  in  this  study  are  listed  in  Table  6.2. 

The  ultimate  goal  is  to  estimate  the  covariate  coefficients,  specifically  the  coefficient  on  the 
past- work-experience  descriptors.  To  do  this  in  Equation  B.2,  one  would  also  have  to  estimate 
the  underlying  hazard  function  8{t )  (Allison,  2010).  However,  the  Cox  PH  model  uses  “partial- 
likelihood  techniques”  (Guo,  Hall-Partyka  and  Gates,  2014,  p.  6)  to  “estimate  the  [covariate] 
coefficients  .  .  .  without  having  to  specify  the  baseline  hazard  function”  (Allison,  2010,  p.  128). 
Specifically,  the  Cox  PH  model  “estimation  method”  (Allison,  2010,  pp.  126-127)  estimates  that 
“the  hazard  for  any  individual  is  a  fixed  proportion  of  the  hazard  for  any  other  individual” 
(Allison,  2010,  p.  127).  See  Equation  B.3. 

Equation  B.  3.  Cox  PH  estimation  method77 

hk(t )  5(t)exp(/?Zfc)  exp  (/?Zfc) 

W)  =  SMexptfZ,)  =  =  '  Zd) 

As  alluded  to  above,  the  underlying  hazard  function,  5(t),  falls  out  of  the  equation  (Guo, 
Hall-Partyka  and  Gates,  2014,  p.  6;  Allison,  2010,  pp.  127-128).  Therefore,  “The  main 


75  This  equation  stems  from  Guo,  Hall-Partyka  and  Gates  (2014)  (p.  6). 

76  This  equation  stems  from  Guo,  Hall-Partyka  and  Gates  (2014)  (p.  6)  and  Allison  (2010)  (p.  127). 

77  This  equation  stems  from  Guo,  Hall-Partyka  and  Gates  (2014)  (p.  6)  and  Allison  (2010)  (p.  127). 
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assumption  [of  the  Cox  PH  model]  is  that  the  hazard  ratio  is  constant  across  time”  (Guo,  Hall- 
Partyka  and  Gates,  2014,  p.  6;  Allison,  2010). 

Because  the  tenure  data  is  recorded  in  integer  years  (one  year,  two  years,  three  years,  etc.), 
there  are  many  individuals  who  are  recorded  as  experiencing  a  loss  from  the  workforce  at  the  end 
of  the  same  tenure  year.  Allison  (2010)  refers  to  these  instances  as  “ties,”  “Tied  Data,”  or  “tied 
event  times”  (p.  142).  Ties  may  worsen  the  estimates  produced  from  the  Cox  PH  model  (Allison, 

no 

2010),  but  the  analyses  include  a  large  number  of  observations  and  cover  a  number  of  years, 
which  should  help  mitigate  this  issue.  Additionally,  the  analyses  use  the  “approximation 
proposed  by  Efron  (1977)”  and,  where  possible,  the  “EXACT”  estimation  techniques  to  produce 
the  best  estimates  using  the  tied  data  (Allison,  2010,  pp.  142-153). 79 

Some  individuals  never  “experience  the  [loss-type(s)]  of  interest”  for  a  particular  analysis 
within  the  years  of  analysis  (Guo,  Hall-Partyka  and  Gates,  2014,  pp.  5-6).  These  individuals  (or 
observations)  are  described  as  “right-censored  observations”  (Goel,  Khanna  and  Kishore,  2010, 
p.  274;  Allison,  2010;  Rich  et  ah,  2010;  Guo,  Hall-Partyka  and  Gates,  2014). 80  For  analyses  on 
retention  in  the  wider  DoD  civilian  workforce,  this  occurs  when  an  individual  is  never  recorded 
as  leaving  the  DoD  civilian  workforce  within  the  years  of  analysis.  For  analyses  on  retention  in 
the  civilian  DAW,  observations  are  right  censored  if  they  are  never  recorded  as  leaving  the 
civilian  DAW  within  the  years  of  analysis  or  if  they  are  marked  as  administratively  transferring 
out  of  the  civilian  DAW.  The  Cox  PH  model  can  handle  these  “censored  data”  (Guo,  Hall- 
Partyka  and  Gates,  2014,  p.  5;  Allison,  2010).  An  observation  that  is  “censored”  at  the  end  of 
year  t  is  included  in  the  calculations  for  year  t  but  not  counted  as  experiencing  the  loss-type  of 
interest  (Allison,  2010,  p.  139).  Then,  the  “censored  observation”  is  not  included  in  the 
calculations  for  the  following  years  (Allison,  2010,  p.  139). 


78 

Allison  (2010)  writes  “When  the  number  of  ties  is  large,  relative  to  the  number  at  risk,  the  approximate  methods 
tend  to  yield  coefficients  that  are  biased  toward  0”  (p.  152). 

79 

The  Cox  PH  models  that  include  propensity  score  weights  use  “Efron’s  approximation”  (Allison,  2010,  p.  146; 
Efron,  1977).  The  models  that  do  not  include  weights  use  the  EXACT  technique.  See  pages  144-148  in  Allison 
(2010)  for  an  explanation  of  “the  EXACT  method.”  Efron’s  approximation  is  from  Efron’s  1977  paper,  Efron 
(1977)  (Allison,  2010). 

80 

The  retention  analysis  is  through  the  end-of-FY  2014.  The  end  of  FY  2014  is  the  last  time  the  analysis  observes 
losses  from  the  workforce.  This  means  that — for  example — an  entrant  who  is  in  the  FY  2002  civilian  DAW  cohort 
and  does  not  experience  a  loss  within  the  years  of  analysis  is  recorded  as  having  a  tenure  of  12  years.  This  individual 
is  right  censored:  He  or  she  could  have  a  longer  tenure,  but  one  cannot  tell  given  the  data  used  (Guo,  Hall-Partyka 
and  Gates,  2014). 

8 1 

All  analyses  on  civilian  DAW  retention  censor  administrative  transfers  out.  Analyses  that  specifically  examine 
substantive  transfers  out  of  the  civilian  DAW  censor  administrative  transfers  out  and  separations.  Analyses  that 
specifically  examine  separations  from  the  civilian  DAW  censor  administrative  transfers  out  and  substantive  transfers 
out. 
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Controlling  for  Differences  in  Observables 

Individuals  who  join  the  civilian  DAW  with  different  past  work  experiences  tend  to  also 
differ  across  the  other  explanatory  variables  listed  in  Table  6.2,  and  these  observable 
characteristics  may  also  influence  retention.  Therefore,  in  order  to  tease  out  the  true  relationship 
between  prior  work  experience  and  retention,  this  analysis  uses  two  methods  to  control  for  these 
differences  in  characteristics:  (1)  propensity-score-based  weighting  (using  RAND’s  Toolkit  for 
Weighting  and  Analysis  of  Nonequivalent  Groups  (TWANG)  (Griffin  et  ah,  2014))  and  (2) 
“multivariate  adjustment”  techniques.  Cox  PH  models  allow  for  weighted  data  and  for  the 
inclusion  of  multivariate  controls  to  adjust  for  differences  between  groups. 

Tests  comparing  retention  trends  between  two  groups  (e.g.,  entrants  with  prior  DoD 
experience  vs.  entrants  with  no  prior  DoD  experience)  use  these  two  control  techniques 

o-i 

together — to  produce  what  are  called  “doubly  robust”  estimates  (Griffin  and  McCaffrey, 
2015b;  Griffin  and  McCaffrey,  2015a,  p.  45;  Griffin  and  McCaffrey,  2015c,  p.  41;  Griffin  and 
McCaffrey,  2015d). 

Propensity  Score  Weights84 

The  propensity-score  weighting  technique  assigns  a  propensity  score  to  each  observation 
(civilian  DAW  entrant,  in  this  case)  included  in  each  particular  analysis.  For  a  test  comparing 
retention  trends  between  entrants  with  and  without  prior  DoD  experience,  the  propensity  score, 
p(x),  is  the  probability  that  an  individual  is  in  the  treatment  group  ( T=1 )  (i.e.,  has  prior  DoD 
experience),  given  that  individual’s  observable  characteristics  (Z)  (Griffin  and  McCaffrey, 
2015b;  Griffin  and  McCaffrey,  2015a;  McCaffrey,  Ridgeway  and  Morral,  2004). 

Equation  B.  4.  The  propensity  score85 
Pi(x )  =  prob(Ti  =  1\  Zf) 

where  T  is  the  treatment  indicator  and  Z  is  the  vector  of  covariates  for  individual  i 

Individuals  with  certain  characteristics  are  more  likely  to  have  prior  DoD  work  experience — the 
propensity  score  captures  this.  Individuals  with  these  particular  characteristics  will  have 


82 

Griffin  and  McCaffrey  (2015d)  use  the  term  “multivariate  adjustment.”  This  description  uses  this  same  term. 

83  The  term  “doubly  robust”  stems  from  Griffin  and  McCaffrey  (2015b),  Griffin  and  McCaffrey  (20 15d),  Griffin 
and  McCaffrey  (2015c)  (p.  41),  Griffin  and  McCaffrey  (2015a)  (p.  45).  This  analysis  also  uses  this  term. 

84 

Griffin  and  McCaffrey  (2015b),  Griffin  and  McCaffrey  (2015a),  and  McCaffrey,  Ridgeway  and  Morral  (2004) 
offer  explanations  of  propensity  scores  and  propensity-score-based  weighting.  The  description  presented  in  this 
study  is  based  on  the  explanations  from  Griffin  and  McCaffrey  (2015b),  Griffin  and  McCaffrey  (2015a),  and 
McCaffrey,  Ridgeway  and  Morral  (2004). 

OC 

This  equation  stems  from  Griffin  and  McCaffrey  (2015a)  (p.  28),  Griffin  and  McCaffrey  (2015b),  and  McCaffrey, 
Ridgeway  and  Morral  (2004)  (p.  405). 
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propensity  score  values  closer  to  one  (Griffin  and  McCaffrey,  2015b;  Griffin  and  McCaffrey, 
2015a,  p.  30). 

RAND’s  TWANG  uses  a  “Data  adaptive,  nonparametric  model”  called  a  Generalized 
Boosted  Model  (GBM)  to  estimate  these  propensity  scores  (Griffin  and  McCaffrey,  2015a,  p.  37; 
McCaffrey,  Ridgeway  and  Morral,  2004,  p.  407;  Griffin  and  McCaffrey,  2015b).  The  GBM  uses 
“Regression  trees”  to  create  and  combine  small  “piecewise”  linear  functions  that — in 
aggregate — estimate  the  propensity  score  curves  (McCaffrey,  Ridgeway  and  Morral,  2004,  p. 
407;  Griffin  and  McCaffrey,  2015a,  p.  37;  Griffin  and  McCaffrey,  2015b).  This  GBM  technique 
is  robust  and  flexible:  It  handles  all  types  of  covariates,  including  categorical  and  continuous 
variables  (Griffin  and  McCaffrey,  2015b;  Griffin  and  McCaffrey,  2015a;  McCaffrey,  Ridgeway 
and  Morral,  2004).  It  uncovers  which  of  these  explanatory  variables  are  most  important  in  terms 
of  predicting  treatment  (e.g.,  whether  an  entrant  has  prior  DoD  experience),  and  the  GBM 
outputs  the  propensity  score  estimations  that  provide  the  best  balance  between  groups  (Griffin 
and  McCaffrey,  2015b;  Griffin  and  McCaffrey,  2015a;  McCaffrey,  Ridgeway  and  Morral,  2004). 
Prior  studies86  “have  shown  that  boosting  outperforms  alternative  methods  in  tenns  of  prediction 
error”  (McCaffrey,  Ridgeway  and  Morral,  2004,  p.  407). 

Once  each  observation  has  an  assigned  propensity  score  based  on  the  observable 
characteristics  (which  are  listed  in  Table  6.2),  one  can  compare  the  propensity  scores  for  the 
individuals  with  and  without  prior  DoD  work  experience.  In  general,  if  the  covariates  used  in  the 
analysis  have  predictive  power  in  terms  of  whether  an  individual  has  prior  DoD  experience, 
individuals  with  DoD  experience  will  tend  to  have  higher  propensity  scores  than  those 
individuals  without  prior  DoD  experience  (Griffin  and  McCaffrey,  2015b;  Griffin  and 
McCaffrey,  2015a;  McCaffrey,  Ridgeway  and  Morral,  2004).  This  makes  sense,  given  Equation 
B.4  (Griffin  and  McCaffrey,  2015b).  Then,  RAND’s  TWANG  applies  weights  to  each 
observation  based  on  these  estimated  propensity  scores  to  balance  the  propensity  score 
distributions  between  these  two  groups  (Griffin  and  McCaffrey,  2015b;  Griffin  and  McCaffrey, 
2015a;  McCaffrey,  Ridgeway  and  Morral,  2004). 

The  models  in  Part  I  of  the  retention  analysis  estimate  the  average  treatment  effect  (ATE). 
For  these  estimates,  RAND’s  TWANG  applies  weights  to  observations  in  both  the  prior-DoD- 
experience  group  and  the  no-prior-DoD-experience  group  to  make  the  average  of  each  group 
look  like  the  overall  average  observation  (i.e.,  the  average  observation  from  the  population  of  all 
entrants  analyzed).  Specifically,  each  observation  (i)  in  the  no-DoD-experience  group  (c) 
receives  a  weight  equal  to  the  inverse  of  one  minus  that  observation’s  propensity  score.  Each 
observation  (i)  in  the  DoD-experienced  group  (T)  receives  a  weight  equal  to  the  inverse  of  that 
observation’s  propensity  score  (McCaffrey,  Ridgeway  and  Morral,  2004;  Griffin  and  McCaffrey, 
2015b;  Griffin  and  McCaffrey,  2015a). 


86 

McCaffrey,  Ridgeway  and  Morral  (2004)  list  the  following  two  studies  (p.  407):  Friedman  (2001)  and  Madigan 
and  Ridgeway  (2004). 
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Equations  B.  5.  Applied  weights  for  ATE87 

=  (l  -  p,M)  w‘T  =  m 

These  weights  bring  the  propensity  score  distributions  for  each  of  the  groups  closer  to  the 
propensity  score  distribution  for  the  overall  sample  (that  includes  all  entrants  being  analyzed) 
(McCaffrey,  Ridgeway  and  Morral,  2004).  Observations  in  the  DoD-experienced  group  with 
higher  propensity  scores  (closer  to  one)  are  weighted  downward,  while  observations  in  the  DoD- 
experienced  group  with  lower  propensity  scores  (closer  to  zero)  are  weighted  upward  and  vice 
versa  for  the  non-DoD-experienced  group  (McCaffrey,  Ridgeway  and  Morral,  2004). 

Balancing  the  groups  on  the  propensity  score  distribution  also  balances  the  groups  on  the 
covariates  used  to  estimate  the  propensity  score — in  terms  of  their  means  and  distributions 
(Griffin  and  McCaffrey,  2015b;  Griffin  and  McCaffrey,  2015a).88 

The  above  explanation  uses  the  comparison  between  DoD-experienced  entrants  and  entrants 
with  no  prior  DoD  experience  as  an  example.  The  same  process  is  used  when  comparing  internal 
and  external  hires. 

Controlling  for  differences  in  observable  characteristics  between  groups  using  propensity 
scores  relies  on  two  major  assumptions:  (1)  All  confounding  factors  must  be  included  in  the 
propensity  score  matching  process,  and  (2)  there  must  be  “overlap”  in  the  observable 
characteristics  between  the  groups  (Griffin  and  McCaffrey,  2015a,  p.  29;  Griffin  and  McCaffrey, 
2015b).  Weighting  the  observations  based  on  propensity  scores  improves  the  balance  between 
groups  for  the  observable  characteristics  included  in  the  matching  process;  however,  these 
weights  will  not  account  for  differences  in  other,  unobserved,  confounding  factors  that  may  exist 
(Morgan  and  Winship,  2007;  Shah  et  ah,  2005;  Zanutto,  2006;  Austin,  2011;  Austin  et  ah,  2005). 
As  such,  this  analysis  includes  an  extensive  list  of  factors  in  the  propensity-score-weighting 
process  (see  Table  6.2). 

“Multivariate  Adjustment”89 

All  Cox  PH  models  displayed  in  Part  I  of  the  retention  analysis  use  multivariate  adjustment 
to  control  for  systematic  differences  between  groups.  This  involves  including  relevant  covariates 
in  the  Cox  PH  models  (Griffin  and  McCaffrey,  20 1 5d).  The  control  factors  (listed  in  Table  6.2) 
are  included  in  the  vector  of  observable  characteristics  (Z),  as  seen  in  Equation  B.2. 

For  retention  tests  comparing  two  groups  (e.g.  entrants  with  prior  DoD  experience  vs.  those 
with  no  prior  DoD  experience),  multivariate  adjustment  is  used  in  combination  with  propensity 

87 

These  equations  stem  from  McCaffrey,  Ridgeway  and  Morral  (2004)  (pp.  405-406),  Griffin  and  McCaffrey 
(2015b),  and  Griffin  and  McCaffrey  (2015a)  (p.  44). 

88 

McCaffrey,  Ridgeway  and  Morral  (2004)  write,  “Rosenbaum  and  Rubin  (1983)  showed  that,  conditional  on  [the 
propensity  score],  all  observed  pretreatment  covariates  are  independent  of  group  assignment  and,  in  large  samples, 
covariates  will  be  distributed  equally  in  both  groups  and  will  not  confound  estimated  treatment  effects”  (p.  404). 

89 

Griffin  and  McCaffrey  (2015d)  use  the  term  “multivariate  adjustment.”  This  description  uses  this  same  term. 
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score  weighting  to  produce  what  are  called  “doubly  robust”  estimates  (Griffin  and  McCaffrey, 
2015b;  Griffin  and  McCaffrey,  2015a,  p.  45;  Griffin  and  McCaffrey,  2015c,  p.  41;  Griffin  and 
McCaffrey,  2015d). 

Retention  tests  that  compare  retention  trends  across  the  prior  career  profiles  only  incorporate 
multivariate  controls.  The  individuals  in  certain  prior  career  profiles  with  DoD  experience  are 
too  fundamentally  different  from  individuals  with  the  No  DoD  Experience  prior  career  profile — 
applied  propensity-score  weights  cannot  bring  these  groups  into  balance.  This  highlights  that 
there  is  not  much  “overlap”  (Griffin  and  McCaffrey,  2015a,  p.  29;  Griffin  and  McCaffrey, 
2015b)  between  the  individuals  with  the  prior-DoD-experience  profiles  and  those  with  the  No 
DoD  Experience  profile,  weakening  the  argument  for  causality.  Given  this,  the  results  from  the 
non-doubly-robust  models  (i.e.,  the  models  that  test  for  retention  differences  across  the  prior 
career  profiles  and  that  incorporate  only  multivariate  adjustment)  are  interpreted  conservatively, 
focusing  on  the  general  direction  of  the  hazard  ratio  estimates  rather  than  on  the  precise  estimate 
number. 

“Doubly  robust  estimation”90 

Griffin  and  McCaffrey  (2015d)  state  that  doubly  robust  models — i.e.,  models  that  use  both 
propensity  score  and  multivariate  techniques  together — are  the  “gold  standard”  in  terms  of 
adjusting  for  observable  confounding  factors. 

First,  including  multivariate  controls  after  weighting  the  data  based  on  propensity  scores 
helps  control  for  differences  between  groups  that  persist  even  after  weighting  (Griffin  and 
McCaffrey,  2015d;  Griffin  and  McCaffrey,  2015c).  Secondly,  propensity  score  weighting 
techniques  rely  on  accurate  propensity  score  estimation,  and  multivariate  regression  techniques 
rely  on  proper  model  specification  (Griffin  and  McCaffrey,  2015d;  Griffin  and  McCaffrey, 
2015c;  Morgan  and  Winship,  2007;  Zanutto,  2006;  Austin,  2011;  Shah  et  ah,  2005).  By 
combining  these  techniques,  i.e.,  first  balancing  on  propensity  scores  then  running  the  model 
with  multivariate  controls,  the  estimates  are  “doubly  robust” — only  one  of  the  control  methods 
has  to  be  done  correctly  to  produce  unbiased  estimates  (Griffin  and  McCaffrey,  2015d;  Griffin 
and  McCaffrey,  2015c,  p.  41). 


90 

The  term  “Doubly  robust  estimation”  stems  from  Griffin  and  McCaffrey  (2015b),  Griffin  and  McCaffrey  (20 1 5d), 
Griffin  and  McCaffrey  (2015c)  (p.  41),  and  Griffin  and  McCaffrey  (2015a). 
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Appendix  C.  Balance  Results  for  DoD-Experienced  Entrants  vs. 
Non-DoD-Experienced  Entrants 


Sections  C.1-C.3  show  how  the  pre -policy  cohort  groups  analyzed  in  Part  I  of  the  retention 
analysis  (see  Table  C.l,  below)  balance,91  in  terms  of  the  observable  characteristics  listed  in 
Table  6.2,  before  and  after  propensity-score  weight  implementation.  Tables  C.l .  1-C.3. 1  list  the 
variable  means  and  standard  deviations  before  and  after  weighting.  And  the  figures  illustrate  how 
effectively  the  propensity-score  weights  bring  the  variable  means  of  the  two  groups  (entrants 
with  prior  DoD  experience  vs.  those  with  no  prior  DoD  experience)  into  balance. 

For  all  three  cohort  groups  analyzed,  the  applied  propensity-score  weights  significantly 
improve  the  balance  between  the  two  groups.  Figures  C.l.l-C.3.1  illustrate  that  the  differences  in 
the  variable  means  between  the  two  groups  decrease  substantially  after  weighting.  In  only  two 
cases — post-weighting — is  the  “Absolute  standard  difference”  in  variable  means  between 
groups  greater  than  0.2,  which  is  the  threshold  referenced  in  the  RAND  TWANG  (Griffin  et  ah, 
2014;  Griffin  and  McCaffrey,  2015d;  Griffin  and  McCaffrey,  2015c).93  These  two  cases  are  in 
the  early  pre-policy  cohort  group  and  are  highlighted  below. 

Figures  C.1.2-C.3.2  show  that  after  applying  propensity-score  weights  some  of  the 
differences  in  variable  means  between  groups  are  no  longer  statistically  significant. 

The  tables  and  figures  presented  in  sections  C.1-C.3  come  from  the  output  of  RAND’s 
TWANG  (Griffin  et  ah,  2014). 

Table  C.l.  Pre-Policy  Cohort  Groups  Analyzed  in  Retention  Analysis  Part  I 

Cohort  Group  Name  FY  Cohorts  Included 

Early  pre-policy  cohort  group  2001-2003 

Later  pre-policy  cohort  group  2004,  2005,  and  2007 

Combination  pre-policy  cohort  group  2001-2005  and  2007 


91 

Griffin  and  McCaffrey  (2015b)  and  Griffin  and  McCaffrey  (2015a)  state  that  balance  assessments  should  consider 
differences  in  variable  means  and  variable  distributions  between  groups.  The  balance  analyses  in  this  study  focus  on 
comparing  variable  means  and  do  not  include  comparing  variable  distributions  because  most  of  the  variables  used 
are  categorical — for  these  variables  the  means  and  distributions  are  essentially  the  same  thing. 

92 

This  phrase  stems  from  Griffin  and  McCaffrey  (2015c)  (p.  28).  The  phrase  is  used  throughout  the  following 
section. 

93 

Some  researchers  use  a  stricter  threshold  of  0. 1  (Austin  and  Stuart,  2015). 
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C.1  Early  Pre-Policy  Cohort  Group 


Table  C.1.1.  Means  and  Standard  Deviations  Before  and  After  Propensity-Score  Weighting 


Unweighted  Weighted 


Balance  Factors 

Mean 

DoD 

SD 

Mean 

No  DoD 

SD 

DoD 

Mean  SD 

Mean 

No  DoD 

SD 

1 

AGE 

43.083 

9.763 

31.77 

10.251 

40.828 

10.756 

38.698 

1 1 .048 

2 

LN(COMPENSATION) 

10.84 

0.318 

10.677 

0.314 

10.808 

0.323 

10.775 

0.317 

3 

RACE:  Other 

0.228 

0.42 

0.235 

0.424 

0.225 

0.418 

0.214 

0.41 

4 

RACE:  White 

0.772 

0.42 

0.765 

0.424 

0.775 

0.418 

0.786 

0.41 

5 

EDUCATION:  BA 

0.379 

0.485 

0.691 

0.462 

0.441 

0.496 

0.496 

0.5 

6 

EDUCATION:  Less  than  BA 

0.412 

0.492 

0.079 

0.27 

0.342 

0.474 

0.278 

0.448 

7 

EDUCATION:  More  than  BA 

0.21 

0.407 

0.23 

0.421 

0.217 

0.412 

0.226 

0.418 

8 

DOD  ORG:  Air  Force 

0.161 

0.368 

0.12 

0.325 

0.153 

0.36 

0.139 

0.346 

9 

DOD  ORG:  Army 

0.443 

0.497 

0.318 

0.466 

0.42 

0.494 

0.413 

0.492 

10 

DOD  ORG:  Navy 

0.293 

0.455 

0.497 

0.5 

0.331 

0.471 

0.372 

0.483 

11 

DOD  ORG:  Other 

0.103 

0.304 

0.065 

0.246 

0.096 

0.294 

0.077 

0.266 

12 

REGION:  East  North  Central 

0.111 

0.315 

0.084 

0.277 

0.106 

0.308 

0.09 

0.287 

13 

REGION:  East  South  Central 

0.088 

0.284 

0.071 

0.257 

0.085 

0.279 

0.071 

0.256 

14 

REGION:  Mid  Atlantic 

0.095 

0.293 

0.116 

0.321 

0.099 

0.299 

0.091 

0.287 

15 

REGION:  Mountain 

0.072 

0.258 

0.045 

0.207 

0.066 

0.248 

0.069 

0.254 

16 

REGION:  New  England 

0.037 

0.188 

0.051 

0.22 

0.039 

0.193 

0.039 

0.193 

17 

REGION:  Other  or  Missing 

0.037 

0.188 

0.007 

0.082 

0.031 

0.172 

0.028 

0.165 

18 

REGION:  Pacific 

0.136 

0.342 

0.198 

0.399 

0.146 

0.353 

0.179 

0.383 

19 

REGION:  South  Atlantic 

0.327 

0.469 

0.378 

0.485 

0.341 

0.474 

0.373 

0.484 

20 

REGION:  West  North  Central 

0.016 

0.124 

0.007 

0.085 

0.014 

0.118 

0.013 

0.115 

21 

REGION:  West  South  Central 

0.082 

0.274 

0.043 

0.203 

0.074 

0.262 

0.047 

0.211 

22 

GENDER:  Female 

0.376 

0.484 

0.311 

0.463 

0.36 

0.48 

0.339 

0.473 

23 

GENDER:  Male 

0.624 

0.484 

0.689 

0.463 

0.64 

0.48 

0.661 

0.473 

24 

POSITION  TYPE:  Administrative 

0.493 

0.5 

0.107 

0.309 

0.411 

0.492 

0.317 

0.465 

25 

POSITION  TYPE:  Other 

0.015 

0.122 

0.01 

0.099 

0.014 

0.117 

0.009 

0.096 

26 

POSITION  TYPE:  Professional 

0.43 

0.495 

0.868 

0.338 

0.522 

0.5 

0.628 

0.483 

27 

POSITION  TYPE:  Technical 

0.063 

0.242 

0.015 

0.122 

0.053 

0.224 

0.047 

0.211 

28 

SUPERVISOR:  All  Other  Positions 

0.896 

0.305 

0.99 

0.1 

0.916 

0.278 

0.936 

0.245 

29 

SUPERVISOR:  Supvisor/Mngr/Ldr 

0.104 

0.305 

0.01 

0.1 

0.084 

0.278 

0.064 

0.245 

30 

ACQ  CAREER  FLD:  Business  (CE  or  FM) 

0.106 

0.307 

0.034 

0.182 

0.09 

0.286 

0.057 

0.232 

31 

ACQ  CAREER  FLD:  Contracting 

0.147 

0.354 

0.162 

0.369 

0.153 

0.36 

0.18 

0.384 

32 

ACQ  CAREER  FLD:  Engineering 

0.216 

0.411 

0.518 

0.5 

0.278 

0.448 

0.337 

0.473 

33 

ACQ  CAREER  FLD:  Facilities  Engineering 

0.022 

0.148 

0.046 

0.209 

0.027 

0.162 

0.036 

0.186 

34 

ACQ  CAREER  FLD:  Information  Tech 

0.086 

0.28 

0.025 

0.157 

0.073 

0.261 

0.057 

0.232 

35 

ACQ  CAREER  FLD:  Life  Cycle  Logistics 

0.119 

0.323 

0.025 

0.157 

0.098 

0.297 

0.077 

0.267 

36 

ACQ  CAREER  FLD:  Other 

0.046 

0.209 

0.015 

0.121 

0.039 

0.194 

0.045 

0.208 

37 

ACQ  CAREER  FLD:  Prod,  Quality  &  Manuf 

0.052 

0.222 

0.02 

0.139 

0.045 

0.207 

0.029 

0.169 

38 

ACQ  CAREER  FLD:  Program  Management 

0.106 

0.307 

0.027 

0.163 

0.09 

0.286 

0.069 

0.254 

39 

ACQ  CAREER  FLD:  Test  and  Eval 

0.038 

0.191 

0.09 

0.286 

0.049 

0.216 

0.055 

0.228 
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40 

41 

42 

43 

44 

45 

46 

47 

48 

49 

50 

51 

52 

S3 

54 

55 

56 

57 

58 

59 

60 


Balance  Factors 

Mean 

Unweighted 

DoD  No  DoD 

SD  Mean  SD 

DoD 

Mean  SD 

Weighted 

No  DoD 

Mean  SD 

ACQ  CAREER  FLD:  Unknown/Missing 

0.064 

0.244 

0.037 

0.189 

0.058 

0.234 

0.058 

0.233 

RETIREMENT  PLAN:  CSRS 

0.263 

0.44 

0.008 

0.088 

0.209 

0.407 

0.129 

0.335 

RETIREMENT  PLAN:  FERS 

0.731 

0.443 

0.987 

0.112 

0.785 

0.411 

0.866 

0.34 

RETIREMENT  PLAN:  Other 

0.006 

0.077 

0.005 

0.071 

0.005 

0.074 

0.005 

0.072 

CAREER  LEVEL:  Entry-Level 

0.172 

0.378 

0.554 

0.497 

0.248 

0.432 

0.304 

0.46 

CAREER  LEVEL:  Mid-Level 

0.738 

0.44 

0.426 

0.494 

0.678 

0.467 

0.66 

0.474 

CAREER  LEVEL:  Sen/Exec 

0.08 

0.271 

0.014 

0.115 

0.066 

0.248 

0.028 

0.164 

CAREER  LEVEL:  Uncategorized 

0.01 

0.099 

0.007 

0.081 

0.009 

0.094 

0.008 

0.089 

COHORT  YEAR:  2001 

0.326 

0.469 

0.25 

0.433 

0.31 

0.463 

0.299 

0.458 

COHORT  YEAR:  2002 

0.37 

0.483 

0.393 

0.488 

0.372 

0.483 

0.397 

0.489 

COHORT  YEAR:  2003 

0.304 

0.46 

0.357 

0.479 

0.318 

0.466 

0.303 

0.46 

YRS  TO  RET  ELIG:  <5  Yrs  to  Go 

0.1 

0.3 

0.025 

0.156 

0.084 

0.277 

0.065 

0.247 

YRS  TO  RET  ELIG:  10  to  19  Yrs  to  Go 

0.399 

0.49 

0.143 

0.35 

0.347 

0.476 

0.315 

0.465 

YRS  TO  RET  ELIG:  20+  YRs  to  Go 

0.264 

0.441 

0.792 

0.406 

0.372 

0.483 

0.481 

0.5 

YRS  TO  RET  ELIG:  5-9  Yrs  to  Go 

0.179 

0.383 

0.036 

0.187 

0.149 

0.357 

0.119 

0.324 

YRS  TO  RET  ELIG:  Ret  Eligible 

0.058 

0.234 

0.004 

0.064 

0.047 

0.211 

0.019 

0.138 

PAY  PLAN:  Broad  GS 

0.84 

0.366 

0.682 

0.466 

0.806 

0.395 

0.789 

0.408 

PAY  PLAN:  Demo 

0.15 

0.357 

0.313 

0.464 

0.185 

0.389 

0.205 

0.404 

PAY  PLAN:  Other 

0.009 

0.095 

0.005 

0.071 

0.008 

0.09 

0.005 

0.072 

HCAP:  Handicapped 

0.094 

0.291 

0.067 

0.25 

0.089 

0.285 

0.09 

0.287 

HCAP:  Not  Handicapped 

0.906 

0.291 

0.933 

0.25 

0.911 

0.285 

0.91 

0.287 

Figure  C.1.1.  Absolute  Standard  Differences  in  Means  Decrease  Substantially  After  Weighting 
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Figure  C.1.2.  After  Weighting,  a  Few  Differences  in  Means  are  no  Longer  Statistically  Significant 
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C.2  Later  Pre-Policy  Cohort  Group 


Table  C.2.1.  Means  and  Standard  Deviations  Before  and  After  Propensity-Score  Weighting 


Balance  Factors 

Mean 

Unweighted 

DoD 

SD  Mean 

No  DoD 

SD 

Mean 

DoD 

SD 

Weighted 

Mean 

No  DoD 

SD 

1 

AGE 

42.28 

10.517 

31.223 

10.017 

39.708 

11.338 

38.018 

11.267 

2 

LN(COMPENSATION) 

10.897 

0.336 

10.665 

0.325 

10.842 

0.348 

10.817 

0.351 

3 

RACE:  Other 

0.227 

0.419 

0.212 

0.409 

0.224 

0.417 

0.236 

0.424 

4 

RACE:  White 

0.773 

0.419 

0.788 

0.409 

0.776 

0.417 

0.764 

0.424 

5 

EDUCATION:  BA 

0.392 

0.488 

0.68 

0.466 

0.462 

0.499 

0.505 

0.5 

6 

EDUCATION:  Less  than  BA 

0.367 

0.482 

0.082 

0.275 

0.298 

0.457 

0.248 

0.432 

7 

EDUCATION:  More  than  BA 

0.241 

0.428 

0.238 

0.426 

0.239 

0.427 

0.247 

0.431 

8 

DOD  ORG:  Air  Force 

0.174 

0.379 

0.089 

0.284 

0.154 

0.361 

0.122 

0.327 

9 

DOD  ORG:  Army 

0.463 

0.499 

0.411 

0.492 

0.447 

0.497 

0.439 

0.496 

10 

DOD  ORG:  Navy 

0.221 

0.415 

0.341 

0.474 

0.252 

0.434 

0.298 

0.457 

11 

DOD  ORG:  Other 

0.142 

0.349 

0.16 

0.366 

0.147 

0.354 

0.141 

0.348 

12 

REGION:  East  North  Central 

0.131 

0.337 

0.112 

0.315 

0.128 

0.334 

0.115 

0.319 

13 

REGION:  East  South  Central 

0.09 

0.286 

0.083 

0.276 

0.088 

0.283 

0.066 

0.247 

14 

REGION:  Mid  Atlantic 

0.09 

0.287 

0.187 

0.39 

0.113 

0.316 

0.115 

0.318 

15 

REGION:  Mountain 

0.073 

0.26 

0.048 

0.213 

0.067 

0.25 

0.07 

0.254 

16 

REGION:  New  England 

0.033 

0.179 

0.037 

0.189 

0.034 

0.181 

0.037 

0.188 

17 

REGION:  Other  or  Missing 

0.031 

0.173 

0.008 

0.09 

0.026 

0.158 

0.024 

0.153 

18 

REGION:  Pacific 

0.111 

0.314 

0.128 

0.334 

0.113 

0.317 

0.133 

0.34 

19 

REGION:  South  Atlantic 

0.346 

0.476 

0.357 

0.479 

0.351 

0.477 

0.373 

0.483 

20 

REGION:  West  North  Central 

0.019 

0.136 

0.009 

0.095 

0.016 

0.127 

0.011 

0.103 

21 

REGION:  West  South  Central 

0.077 

0.266 

0.031 

0.173 

0.065 

0.247 

0.058 

0.234 

22 

GENDER:  Female 

0.35 

0.477 

0.366 

0.482 

0.353 

0.478 

0.379 

0.485 

23 

GENDER:  Male 

0.65 

0.477 

0.634 

0.482 

0.647 

0.478 

0.621 

0.485 

24 

POSITION  TYPE:  Administrative 

0.479 

0.5 

0.154 

0.361 

0.401 

0.49 

0.328 

0.47 

25 

POSITION  TYPE:  Other 

0.012 

0.111 

0.009 

0.092 

0.011 

0.105 

0.014 

0.119 

26 

POSITION  TYPE:  Professional 

0.44 

0.496 

0.816 

0.387 

0.53 

0.499 

0.613 

0.487 

27 

POSITION  TYPE:  Technical 

0.069 

0.253 

0.021 

0.144 

0.058 

0.233 

0.044 

0.206 

28 

SUPERVISOR:  All  Other  Positions 

0.892 

0.311 

0.99 

0.102 

0.915 

0.279 

0.902 

0.298 

29 

SUPERVISOR:  Supvisor/Mngr/Ldr 

0.108 

0.311 

0.01 

0.102 

0.085 

0.279 

0.098 

0.298 

30 

ACQ  CAREER  FLD:  Business  (CE  or  FM) 

0.084 

0.278 

0.049 

0.217 

0.077 

0.266 

0.078 

0.268 

31 

ACQ  CAREER  FLD:  Contracting 

0.157 

0.364 

0.198 

0.398 

0.165 

0.371 

0.206 

0.404 

32 

ACQ  CAREER  FLD:  Engineering 

0.151 

0.358 

0.388 

0.487 

0.209 

0.406 

0.245 

0.43 

33 

ACQ  CAREER  FLD:  Facilities  Engineering 

0.085 

0.279 

0.045 

0.207 

0.075 

0.264 

0.079 

0.27 

34 

ACQ  CAREER  FLD:  Information  Tech 

0.045 

0.208 

0.025 

0.155 

0.04 

0.197 

0.035 

0.183 

35 

ACQ  CAREER  FLD:  Life  Cycle  Logistics 

0.157 

0.364 

0.047 

0.211 

0.13 

0.337 

0.086 

0.28 

36 

ACQ  CAREER  FLD:  Other 

0.037 

0.189 

0.013 

0.112 

0.031 

0.173 

0.027 

0.161 

37 

ACQ  CAREER  FLD:  Prod,  Quality  &  Manuf 

0.085 

0.279 

0.04 

0.196 

0.074 

0.261 

0.05 

0.218 

38 

ACQ  CAREER  FLD:  Program  Management 

0.092 

0.288 

0.027 

0.161 

0.076 

0.265 

0.07 

0.255 

39 

ACQ  CAREER  FLD:  Test  and  Eval 

0.035 

0.185 

0.063 

0.242 

0.043 

0.203 

0.044 

0.205 

98 


40 

41 
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43 

44 

45 

46 

47 

48 

49 

50 
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52 

53 

54 
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58 
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Balance  Factors 

Mean 

Unweighted 

DoD  No  DoD 

SD  Mean  SD 

Mean 

\ 

DoD 

SD 

l/Veighted 

No  DoD 

Mean  SD 

ACQ  CAREER  FLD:  Unknown/Missing 

0.071 

0.256 

0.106 

0.308 

0.08 

0.271 

0.083 

0.275 

RETIREMENT  PLAN:  CSRS 

0.149 

0.356 

0.004 

0.063 

0.114 

0.318 

0.092 

0.288 

RETIREMENT  PLAN:  FERS 

0.843 

0.364 

0.987 

0.112 

0.878 

0.327 

0.9 

0.3 

RETIREMENT  PLAN:  Other 

0.008 

0.09 

0.009 

0.093 

0.008 

0.089 

0.009 

0.093 

CAREER  LEVEL:  Entry-Level 

0.203 

0.402 

0.6 

0.49 

0.297 

0.457 

0.343 

0.475 

CAREER  LEVEL:  Mid-Level 

0.691 

0.462 

0.372 

0.483 

0.616 

0.486 

0.578 

0.494 

CAREER  LEVEL:  Sen/Exec 

0.095 

0.294 

0.016 

0.126 

0.076 

0.266 

0.069 

0.254 

CAREER  LEVEL:  Uncategorized 

0.01 

0.099 

0.011 

0.107 

0.01 

0.101 

0.009 

0.096 

COHORT  YEAR:  2004 

0.37 

0.483 

0.329 

0.47 

0.362 

0.481 

0.354 

0.478 

COHORT  YEAR:  2005 

0.358 

0.479 

0.321 

0.467 

0.344 

0.475 

0.353 

0.478 

COHORT  YEAR:  2007 

0.272 

0.445 

0.35 

0.477 

0.294 

0.456 

0.293 

0.455 

YRS  TO  RET  ELIG:  <5  Yrs  to  Go 

0.094 

0.291 

0.024 

0.152 

0.077 

0.266 

0.055 

0.227 

YRS  TO  RET  ELIG:  10  to  19  Yrs  to  Go 

0.38 

0.485 

0.132 

0.338 

0.322 

0.467 

0.287 

0.453 

YRS  TO  RET  ELIG:  20+  YRs  to  Go 

0.322 

0.467 

0.808 

0.394 

0.438 

0.496 

0.518 

0.5 

YRS  TO  RET  ELIG:  5-9  Yrs  to  Go 

0.15 

0.357 

0.034 

0.18 

0.123 

0.328 

0.112 

0.315 

YRS  TO  RET  ELIG:  Ret  Eligible 

0.053 

0.225 

0.004 

0.06 

0.041 

0.199 

0.028 

0.165 

PAY  PLAN:  Broad  GS 

0.801 

0.399 

0.716 

0.451 

0.781 

0.414 

0.763 

0.425 

PAY  PLAN:  Demo 

0.141 

0.348 

0.244 

0.429 

0.165 

0.371 

0.194 

0.395 

PAY  PLAN:  NSPS 

0.049 

0.215 

0.03 

0.171 

0.045 

0.207 

0.035 

0.184 

PAY  PLAN:  Other 

0.01 

0.098 

0.01 

0.1 

0.01 

0.098 

0.008 

0.089 

HCAP:  Handicapped 

0.089 

0.284 

0.051 

0.22 

0.08 

0.271 

0.072 

0.259 

HCAP:  Not  Handicapped 

0.911 

0.284 

0.949 

0.22 

0.92 

0.271 

0.928 

0.259 
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Figure  C.2.1.  Absolute  Standard  Differences  in  Means  Decrease  Substantially  After  Weighting 


Figure  C.2.2.  After  Weighting,  Several  Differences  in  Means  are  no  Longer  Statistically  Significant 
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C.3  Combination  Pre-Policy  Cohort  Group 


Table  C.3.1.  Means  and  Standard  Deviations  Before  and  After  Propensity-Score  Weighting 


Balance  Factors 


AGE 

LN(COMPENSATION) 

RACE:  Other 
RACE:  White 
EDUCATION:  BA 
EDUCATION:  Less  than  BA 
EDUCATION:  More  than  BA 
DOD  ORG:  Air  Force 
DOD  ORG:  Army 
DOD  ORG:  Navy 
DOD  ORG:  Other 
REGION:  East  North  Central 
REGION:  East  South  Central 
REGION:  Mid  Atlantic 
REGION:  Mountain 
REGION:  New  England 
REGION:  Other  or  Missing 
REGION:  Pacific 
REGION:  South  Atlantic 
REGION:  West  North  Central 
REGION:  West  South  Central 
GENDER:  Female 
GENDER:  Male 

POSITION  TYPE:  Administrative 
POSITION  TYPE:  Other 
POSITION  TYPE:  Professional 
POSITION  TYPE:  Technical 
SUPERVISOR:  All  Other  Positions 
SUPERVISOR:  Supvisor/Mngr/Ldr 
ACQ  CAREER  FLD:  Business  (CE  or  FM) 
ACQ  CAREER  FLD:  Contracting 
ACQ  CAREER  FLD:  Engineering 
ACQ  CAREER  FLD:  Facilities  Engineering 
ACQ  CAREER  FLD:  Information  Tech 
ACQ  CAREER  FLD:  Life  Cycle  Logistics 
ACQ  CAREER  FLD:  Other 
ACQ  CAREER  FLD:  Prod,  Quality  &  Manuf 
ACQ  CAREER  FLD:  Program  Management 
ACQ  CAREER  FLD:  Test  and  Eval 


Unweighted 


Mean 

DoD 

SD 

Mean 

No  DoD 

SD 

42.68 

10.156 

31.473 

10.128 

10.869 

0.329 

10.67 

0.32 

0.228 

0.419 

0.223 

0.416 

0.772 

0.419 

0.777 

0.416 

0.385 

0.487 

0.685 

0.465 

0.389 

0.488 

0.081 

0.273 

0.225 

0.418 

0.234 

0.423 

0.168 

0.374 

0.103 

0.304 

0.453 

0.498 

0.369 

0.482 

0.257 

0.437 

0.412 

0.492 

0.122 

0.328 

0.116 

0.321 

0.121 

0.326 

0.099 

0.299 

0.089 

0.285 

0.078 

0.267 

0.093 

0.29 

0.155 

0.362 

0.072 

0.259 

0.046 

0.21 

0.035 

0.183 

0.043 

0.204 

0.034 

0.181 

0.008 

0.087 

0.123 

0.329 

0.16 

0.367 

0.336 

0.472 

0.366 

0.482 

0.017 

0.13 

0.008 

0.09 

0.079 

0.27 

0.036 

0.187 

0.363 

0.481 

0.341 

0.474 

0.637 

0.481 

0.659 

0.474 

0.486 

0.5 

0.132 

0.339 

0.014 

0.116 

0.009 

0.095 

0.435 

0.496 

0.84 

0.367 

0.066 

0.248 

0.018 

0.134 

0.894 

0.308 

0.99 

0.101 

0.106 

0.308 

0.01 

0.101 

0.095 

0.293 

0.042 

0.202 

0.152 

0.359 

0.181 

0.385 

0.183 

0.387 

0.448 

0.497 

0.054 

0.226 

0.045 

0.208 

0.065 

0.247 

0.025 

0.156 

0.138 

0.345 

0.037 

0.189 

0.041 

0.199 

0.014 

0.116 

0.069 

0.253 

0.031 

0.173 

0.099 

0.298 

0.027 

0.162 

0.037 

0.188 

0.075 

0.264 

Weighted 


Mean 

DoD 

SD 

Mean 

No  DoD 

SD 

40.162 

11.104 

38.681 

11.229 

10.824 

0.337 

10.805 

0.344 

0.226 

0.418 

0.223 

0.416 

0.774 

0.418 

0.777 

0.416 

0.453 

0.498 

0.49 

0.5 

0.318 

0.466 

0.272 

0.445 

0.229 

0.42 

0.237 

0.425 

0.153 

0.36 

0.125 

0.33 

0.433 

0.495 

0.426 

0.495 

0.292 

0.455 

0.325 

0.469 

0.122 

0.327 

0.123 

0.329 

0.117 

0.321 

0.107 

0.309 

0.087 

0.282 

0.074 

0.261 

0.106 

0.308 

0.101 

0.301 

0.066 

0.248 

0.07 

0.255 

0.037 

0.189 

0.039 

0.193 

0.028 

0.165 

0.026 

0.158 

0.13 

0.336 

0.147 

0.354 

0.345 

0.475 

0.37 

0.483 

0.015 

0.123 

0.013 

0.112 

0.069 

0.253 

0.054 

0.226 

0.357 

0.479 

0.369 

0.483 

0.643 

0.479 

0.631 

0.483 

0.403 

0.491 

0.322 

0.467 

0.013 

0.112 

0.011 

0.105 

0.529 

0.499 

0.614 

0.487 

0.055 

0.228 

0.053 

0.223 

0.916 

0.277 

0.913 

0.282 

0.084 

0.277 

0.087 

0.282 

0.084 

0.277 

0.066 

0.248 

0.158 

0.365 

0.199 

0.399 

0.244 

0.43 

0.279 

0.448 

0.052 

0.221 

0.053 

0.225 

0.056 

0.23 

0.045 

0.207 

0.114 

0.318 

0.076 

0.264 

0.035 

0.184 

0.046 

0.209 

0.06 

0.237 

0.042 

0.2 

0.082 

0.274 

0.08 

0.272 

0.046 

0.21 

0.049 

0.215 
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40 

41 

42 

43 

44 

45 

46 

47 

48 

49 

50 

51 

52 

53 

54 

55 

56 

57 

58 

59 

60 

61 

62 

63 

64 


Balance  Factors 


ACQ  CAREER  FLD:  Unknown/Missing 

RETIREMENT  PLAN:  CSRS 

RETIREMENT  PLAN:  FERS 

RETIREMENT  PLAN:  Other 

CAREER  LEVEL:  Entry-Level 

CAREER  LEVEL:  Mid-Level 

CAREER  LEVEL:  Sen/Exec 

CAREER  LEVEL:  Uncategorized 

COHORT  YEAR:  2001 

COHORT  YEAR:  2002 

COHORT  YEAR:  2003 

COHORT  YEAR:  2004 

COHORT  YEAR:  2005 

COHORT  YEAR:  2007 

YRS  TO  RET  ELIG:  <5  Yrs  to  Go 

YRS  TO  RET  ELIG:  10  to  19  Yrs  to  Go 

YRS  TO  RET  ELIG:  20+  YRs  to  Go 

YRS  TO  RET  ELIG:  5-9  Yrs  to  Go 

YRS  TO  RET  ELIG:  Ret  Eligible 

PAY  PLAN:  Broad  GS 

PAY  PLAN:  Demo 

PAY  PLAN:  NSPS 

PAY  PLAN:  Other 

HCAP:  Handicapped 

HCAP:  Not  Handicapped 


Unweighted 

DoD  No  DoD 


Mean 

SD 

Mean 

SD 

0.067 

0.25 

0.075 

0.263 

0.206 

0.404 

0.006 

0.075 

0.787 

0.409 

0.987 

0.112 

0.007 

0.083 

0.007 

0.083 

0.188 

0.391 

0.579 

0.494 

0.715 

0.452 

0.397 

0.489 

0.088 

0.283 

0.015 

0.121 

0.01 

0.099 

0.009 

0.096 

0.163 

0.369 

0.114 

0.318 

0.184 

0.388 

0.18 

0.384 

0.151 

0.358 

0.163 

0.37 

0.186 

0.389 

0.179 

0.383 

0.18 

0.384 

0.174 

0.379 

0.137 

0.343 

0.19 

0.393 

0.097 

0.296 

0.024 

0.154 

0.39 

0.488 

0.137 

0.343 

0.294 

0.455 

0.8 

0.4 

0.165 

0.371 

0.035 

0.184 

0.056 

0.229 

0.004 

0.062 

0.821 

0.384 

0.7 

0.458 

0.145 

0.353 

0.275 

0.447 

0.025 

0.155 

0.016 

0.127 

0.009 

0.097 

0.008 

0.088 

0.091 

0.288 

0.058 

0.235 

0.909 

0.288 

0.942 

0.235 

Weighted 


Mean 

DoD 

SD 

Mean 

No  DoD 

SD 

0.069 

0.254 

0.066 

0.249 

0.159 

0.366 

0.111 

0.315 

0.834 

0.372 

0.881 

0.324 

0.007 

0.082 

0.008 

0.088 

0.275 

0.447 

0.318 

0.466 

0.644 

0.479 

0.609 

0.488 

0.071 

0.257 

0.064 

0.245 

0.01 

0.099 

0.009 

0.094 

0.15 

0.357 

0.134 

0.341 

0.18 

0.384 

0.187 

0.39 

0.152 

0.359 

0.134 

0.34 

0.188 

0.391 

0.194 

0.395 

0.178 

0.382 

0.194 

0.395 

0.152 

0.359 

0.158 

0.365 

0.08 

0.271 

0.065 

0.247 

0.332 

0.471 

0.305 

0.46 

0.41 

0.492 

0.486 

0.5 

0.135 

0.341 

0.116 

0.32 

0.044 

0.205 

0.028 

0.165 

0.792 

0.406 

0.767 

0.423 

0.177 

0.381 

0.203 

0.402 

0.023 

0.149 

0.022 

0.146 

0.009 

0.095 

0.008 

0.091 

0.085 

0.278 

0.082 

0.274 

0.915 

0.278 

0.918 

0.274 
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Figure  C.3.1.  Absolute  Standard  Differences  in  Means  Decrease  Substantially  after  Weighting 


Unweighted  Weighted 


Figure  C.3.2.  After  Weighting,  a  Few  Differences  in  Means  are  No  Longer  Statistically  Significant 
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Appendix  D.  Balance  Results  for  Internal  Hires  vs.  External  Hires 


Sections  D.1-D.3  show  how  the  pre-policy  cohort  groups  analyzed  in  Part  I  of  the  retention 
analysis  (see  Table  D.l,  below)  balance,94  in  terms  of  the  observable  characteristics  listed  in 
Table  6.2,  before  and  after  propensity-score -weight  implementation.  Tables  D.l.l-D.3.1  list  the 
variable  means  and  standard  deviations  before  and  after  weighting.  And  the  figures  illustrate  how 
effectively  the  propensity-score  weights  bring  the  variable  means  of  the  two  groups  (internal 
hires  vs.  external  hires)  into  balance. 

For  all  three  cohort  groups  analyzed,  the  applied  propensity-score  weights  significantly 
improve  the  balance  between  the  two  groups.  Figures  D.l.l-D.3.1  illustrate  that  the  differences 
in  the  variable  means  between  the  two  groups  decrease  substantially  after  weighting.  In  no 
cases — post-weighting — is  the  “Absolute  standard  difference”95  in  variable  means  between 
groups  greater  than  0.2,  which  is  the  threshold  referenced  in  the  RAND  TWANG  (Griffin  et  ah, 
2014;  Griffin  and  McCaffrey,  2015d;  Griffin  and  McCaffrey,  2015c).96 

Figures  D.1.2-D.3.2  show  that  after  applying  propensity-score  weights,  some  of  the 
differences  in  variable  means  between  groups  are  no  longer  statistically  significant. 

The  tables  and  figures  presented  in  sections  D.1-D.3  come  from  the  output  of  RAND’s 
TWANG  (Griffin  et  ah,  2014). 


Table  D.l.  Pre-Policy  Cohort  Groups  Analyzed  in  Retention  Analysis  Part  I 

Cohort  Group  Name  FY  Cohorts  Included 

Early  Pre-policy  cohort  group  2001-2003 

Later  pre-policy  cohort  group  2004,  2005,  and  2007 

Combination  pre-policy  cohort  group  2001-2005  and  2007 


94 

Griffin  and  McCaffrey  (2015b)  and  Griffin  and  McCaffrey  (2015a)  state  that  balance  assessments  should  consider 
differences  in  variable  means  and  variable  distributions  between  groups.  The  balance  analyses  in  this  study  focus  on 
comparing  variable  means  and  do  not  include  comparing  variable  distributions  because  most  of  the  variables  used 
are  categorical — for  these  variables  the  means  and  distributions  are  essentially  the  same  thing. 

95 

This  phrase  stems  from  Griffin  and  McCaffrey  (2015c)  (p.  28).  The  phrase  is  used  throughout  the  following 
section. 

96 

Some  researchers  use  a  stricter  threshold  of  0. 1  (Austin  and  Stuart,  2015). 
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D.1  Early  Pre-Policy  Cohort  Group 


Table  D.1.1.  Means  and  Standard  Deviations  Before  and  After  Propensity-Score  Weighting 


Balance  Factors 

Unweighted 

Internal  Hire  External  Hire 

Mean  SD  Mean  SD 

Weighted 

Internal  Hire  External  Hire 

Mean  SD  Mean  SD 

1 

AGE 

43.161 

10.262 

37.018 

10.928 

40.669 

10.921 

39.738 

10.965 

2 

LN(COMPENSATION) 

10.832 

0.306 

10.763 

0.341 

10.805 

0.32 

10.784 

0.326 

3 

RACE:  Other 

0.243 

0.429 

0.215 

0.411 

0.225 

0.418 

0.22 

0.414 

4 

RACE:  White 

0.757 

0.429 

0.785 

0.411 

0.775 

0.418 

0.78 

0.414 

5 

EDUCATION:  BA 

0.37 

0.483 

0.554 

0.497 

0.441 

0.497 

0.467 

0.499 

6 

EDUCATION:  Less  than  BA 

0.464 

0.499 

0.174 

0.379 

0.349 

0.477 

0.313 

0.464 

7 

EDUCATION:  More  than  BA 

0.166 

0.372 

0.273 

0.445 

0.21 

0.407 

0.221 

0.415 

8 

DOD  ORG:  Air  Force 

0.144 

0.351 

0.16 

0.366 

0.157 

0.363 

0.157 

0.364 

9 

DOD  ORG:  Army 

0.495 

0.5 

0.315 

0.465 

0.427 

0.495 

0.394 

0.489 

10 

DOD  ORG:  Navy 

0.257 

0.437 

0.444 

0.497 

0.314 

0.464 

0.355 

0.479 

11 

DOD  ORG:  Other 

0.104 

0.306 

0.081 

0.273 

0.103 

0.303 

0.094 

0.292 

12 

REGION:  East  North  Central 

0.122 

0.328 

0.084 

0.278 

0.103 

0.304 

0.089 

0.285 

13 

REGION:  East  South  Central 

0.097 

0.296 

0.069 

0.254 

0.081 

0.272 

0.069 

0.253 

14 

REGION:  Mid  Atlantic 

0.109 

0.312 

0.089 

0.285 

0.1 

0.3 

0.105 

0.306 

15 

REGION:  Mountain 

0.065 

0.247 

0.065 

0.247 

0.068 

0.252 

0.067 

0.25 

16 

REGION:  New  England 

0.034 

0.181 

0.047 

0.213 

0.036 

0.187 

0.045 

0.208 

17 

REGION:  Other  or  Missing 

0.04 

0.195 

0.017 

0.129 

0.032 

0.177 

0.027 

0.161 

18 

REGION:  Pacific 

0.131 

0.338 

0.174 

0.379 

0.148 

0.355 

0.164 

0.37 

19 

REGION:  South  Atlantic 

0.293 

0.455 

0.393 

0.488 

0.341 

0.474 

0.352 

0.478 

20 

REGION:  West  North  Central 

0.017 

0.128 

0.01 

0.1 

0.015 

0.121 

0.014 

0.118 

21 

REGION:  West  South  Central 

0.091 

0.288 

0.051 

0.219 

0.075 

0.264 

0.069 

0.254 

22 

GENDER:  Female 

0.447 

0.497 

0.258 

0.438 

0.368 

0.482 

0.357 

0.479 

23 

GENDER:  Male 

0.553 

0.497 

0.742 

0.438 

0.632 

0.482 

0.643 

0.479 

24 

POSITION  TYPE:  Administrative 

0.535 

0.499 

0.239 

0.426 

0.413 

0.492 

0.357 

0.479 

25 

POSITION  TYPE:  Other 

0.015 

0.121 

0.013 

0.112 

0.014 

0.119 

0.016 

0.125 

26 

POSITION  TYPE:  Professional 

0.38 

0.485 

0.72 

0.449 

0.519 

0.5 

0.57 

0.495 

27 

POSITION  TYPE:  Technical 

0.07 

0.255 

0.029 

0.168 

0.053 

0.225 

0.058 

0.233 

28 

SUPERVISOR:  All  Other  Positions 

0.886 

0.318 

0.958 

0.2 

0.914 

0.28 

0.932 

0.252 

29 

SUPERVISOR:  Supvisor/Mngr/Ldr 

0.114 

0.318 

0.042 

0.2 

0.086 

0.28 

0.068 

0.252 

30 

ACQ  CAREER  FLD:  Business  (CE  or  FM) 

0.13 

0.336 

0.039 

0.194 

0.094 

0.292 

0.096 

0.295 

31 

ACQ  CAREER  FLD:  Contracting 

0.123 

0.329 

0.184 

0.387 

0.154 

0.361 

0.163 

0.369 

32 

ACQ  CAREER  FLD:  Engineering 

0.199 

0.4 

0.394 

0.489 

0.27 

0.444 

0.305 

0.461 

33 

ACQ  CAREER  FLD:  Facilities  Engineering 

0.021 

0.144 

0.036 

0.187 

0.027 

0.161 

0.03 

0.17 

34 

ACQ  CAREER  FLD:  Information  Tech 

0.098 

0.297 

0.04 

0.195 

0.076 

0.265 

0.057 

0.232 

35 

ACQ  CAREER  FLD:  Life  Cycle  Logistics 

0.12 

0.325 

0.068 

0.252 

0.097 

0.296 

0.083 

0.276 

36 

ACQ  CAREER  FLD:  Other 

0.051 

0.22 

0.023 

0.15 

0.04 

0.196 

0.046 

0.21 

37 

ACQ  CAREER  FLD:  Prod,  Quality  &  Manuf 

0.056 

0.23 

0.03 

0.171 

0.044 

0.206 

0.037 

0.189 

38 

ACQ  CAREER  FLD:  Program  Management 

0.105 

0.307 

0.065 

0.246 

0.089 

0.285 

0.07 

0.255 

39 

ACQ  CAREER  FLD:  Test  and  Eval 

0.029 

0.168 

0.076 

0.265 

0.047 

0.212 

0.055 

0.228 

105 


40 

41 

42 

43 

44 

45 

46 

47 

48 

49 

50 

51 

52 

53 

54 

55 

56 

57 

58 

59 

60 


Unweighted  Weighted 


Balance  Factors  Internal  Hire 

Mean  SD 


ACQ  CAREER  FLD:  Unknown/Missing 

0.068 

0.251 

RETIREMENT  PLAN:  CSRS 

0.33 

0.47 

RETIREMENT  PLAN:  FERS 

0.668 

0.471 

RETIREMENT  PLAN:  Other 

0.002 

0.043 

CAREER  LEVEL:  Entry-Level 

0.168 

0.374 

CAREER  LEVEL:  Mid-Level 

0.744 

0.436 

CAREER  LEVEL:  Sen/Exec 

0.083 

0.276 

CAREER  LEVEL:  Uncategorized 

0.005 

0.068 

COHORT  YEAR:  2001 

0.353 

0.478 

COHORT  YEAR:  2002 

0.376 

0.484 

COHORT  YEAR:  2003 

0.271 

0.445 

YRS  TO  RET  ELIG:  <5  Yrs  to  Go 

0.12 

0.325 

YRS  TO  RET  ELIG:  10  to  19  Yrs  to  Go 

0.353 

0.478 

YRS  TO  RET  ELIG:  20+  YRs  to  Go 

0.252 

0.434 

YRS  TO  RET  ELIG:  5-9  Yrs  to  Go 

0.201 

0.401 

YRS  TO  RET  ELIG:  Ret  Eligible 

0.074 

0.261 

PAY  PLAN:  Broad  GS 

0.871 

0.336 

PAY  PLAN:  Demo 

0.126 

0.332 

PAY  PLAN:  Other 

0.003 

0.059 

HCAP:  Handicapped 

0.085 

0.278 

HCAP:  Not  Handicapped 

0.915 

0.278 

External  Hire  Internal  Hire  External  Hire 


Mean 

SD 

Mean 

SD 

Mean 

SD 

0.045 

0.207 

0.062 

0.241 

0.058 

0.233 

0.049 

0.215 

0.219 

0.414 

0.171 

0.377 

0.941 

0.235 

0.778 

0.416 

0.822 

0.382 

0.01 

0.101 

0.003 

0.057 

0.006 

0.08 

0.379 

0.485 

0.243 

0.429 

0.285 

0.452 

0.566 

0.496 

0.685 

0.465 

0.65 

0.477 

0.041 

0.198 

0.066 

0.248 

0.055 

0.227 

0.014 

0.119 

0.007 

0.082 

0.01 

0.098 

0.254 

0.435 

0.318 

0.466 

0.29 

0.454 

0.375 

0.484 

0.364 

0.481 

0.377 

0.485 

0.37 

0.483 

0.318 

0.466 

0.333 

0.471 

0.037 

0.188 

0.086 

0.281 

0.065 

0.247 

0.317 

0.465 

0.345 

0.475 

0.337 

0.473 

0.558 

0.497 

0.367 

0.482 

0.421 

0.494 

0.077 

0.267 

0.153 

0.36 

0.14 

0.347 

0.011 

0.105 

0.049 

0.216 

0.038 

0.191 

0.721 

0.448 

0.817 

0.387 

0.793 

0.406 

0.265 

0.441 

0.177 

0.382 

0.198 

0.399 

0.014 

0.116 

0.006 

0.077 

0.009 

0.095 

0.09 

0.286 

0.093 

0.29 

0.088 

0.283 

0.91 

0.286 

0.907 

0.29 

0.912 

0.283 
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Figure  D.1.2.  After  Weighting,  Two  Differences  in  Means  are  no  Longer  Statistically  Significant 
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D.2  Later  Pre-Policy  Cohort  Group 


Table  D.2.1.  Means  and  Standard  Deviations  Before  and  After  Propensity-Score  Weighting 


Balance  Factors 

Unweighted 

Internal  Hire  External  Hire 

Mean  SD  Mean  SD 

Internal  Hire 

Mean  SD 

Weighted 

External  Hire 

Mean  SD 

1 

AGE 

41.787 

11.372 

36.815 

11.072 

39.429 

11.545 

39.111 

11.349 

2 

LN(COMPENSATION) 

10.874 

0.32 

10.795 

0.37 

10.841 

0.346 

10.828 

0.353 

3 

RACE:  Other 

0.237 

0.425 

0.21 

0.407 

0.224 

0.417 

0.22 

0.414 

4 

RACE:  White 

0.763 

0.425 

0.79 

0.407 

0.776 

0.417 

0.78 

0.414 

5 

EDUCATION:  BA 

0.402 

0.49 

0.537 

0.499 

0.47 

0.499 

0.476 

0.499 

6 

EDUCATION:  Less  than  BA 

0.399 

0.49 

0.183 

0.387 

0.295 

0.456 

0.279 

0.448 

7 

EDUCATION:  More  than  BA 

0.199 

0.399 

0.28 

0.449 

0.236 

0.425 

0.245 

0.43 

8 

DOD  ORG:  Air  Force 

0.164 

0.37 

0.139 

0.346 

0.154 

0.361 

0.142 

0.349 

9 

DOD  ORG:  Army 

0.499 

0.5 

0.4 

0.49 

0.45 

0.498 

0.443 

0.497 

10 

DOD  ORG:  Navy 

0.191 

0.393 

0.314 

0.464 

0.254 

0.435 

0.261 

0.439 

11 

DOD  ORG:  Other 

0.146 

0.353 

0.148 

0.355 

0.142 

0.349 

0.155 

0.361 

12 

REGION:  East  North  Central 

0.145 

0.352 

0.107 

0.309 

0.125 

0.331 

0.124 

0.33 

13 

REGION:  East  South  Central 

0.094 

0.292 

0.082 

0.274 

0.084 

0.278 

0.079 

0.269 

14 

REGION:  Mid  Atlantic 

0.103 

0.304 

0.13 

0.337 

0.115 

0.319 

0.117 

0.321 

15 

REGION:  Mountain 

0.069 

0.254 

0.063 

0.243 

0.066 

0.248 

0.069 

0.253 

16 

REGION:  New  England 

0.032 

0.175 

0.036 

0.188 

0.032 

0.177 

0.034 

0.182 

17 

REGION:  Other  or  Missing 

0.031 

0.173 

0.019 

0.136 

0.026 

0.159 

0.028 

0.165 

18 

REGION:  Pacific 

0.106 

0.307 

0.125 

0.331 

0.117 

0.321 

0.123 

0.328 

19 

REGION:  South  Atlantic 

0.321 

0.467 

0.376 

0.484 

0.349 

0.477 

0.351 

0.477 

20 

REGION:  West  North  Central 

0.02 

0.139 

0.013 

0.112 

0.017 

0.128 

0.015 

0.122 

21 

REGION:  West  South  Central 

0.08 

0.272 

0.048 

0.214 

0.068 

0.252 

0.061 

0.239 

22 

GENDER:  Female 

0.416 

0.493 

0.294 

0.456 

0.358 

0.479 

0.348 

0.476 

23 

GENDER:  Male 

0.584 

0.493 

0.706 

0.456 

0.642 

0.479 

0.652 

0.476 

24 

POSITION  TYPE:  Administrative 

0.496 

0.5 

0.288 

0.453 

0.4 

0.49 

0.373 

0.484 

25 

POSITION  TYPE:  Other 

0.012 

0.11 

0.01 

0.102 

0.012 

0.108 

0.011 

0.103 

26 

POSITION  TYPE:  Professional 

0.417 

0.493 

0.664 

0.472 

0.532 

0.499 

0.558 

0.497 

27 

POSITION  TYPE:  Technical 

0.075 

0.263 

0.037 

0.19 

0.057 

0.231 

0.058 

0.233 

28 

SUPERVISOR:  All  Other  Positions 

0.888 

0.315 

0.947 

0.223 

0.915 

0.278 

0.922 

0.268 

29 

SUPERVISOR:  Supvisor/Mngr/Ldr 

0.112 

0.315 

0.053 

0.223 

0.085 

0.278 

0.078 

0.268 

30 

ACQ  CAREER  FLD:  Business  (CE  or  FM) 

0.1 

0.3 

0.05 

0.218 

0.078 

0.268 

0.082 

0.275 

31 

ACQ  CAREER  FLD:  Contracting 

0.119 

0.323 

0.215 

0.411 

0.166 

0.372 

0.175 

0.38 

32 

ACQ  CAREER  FLD:  Engineering 

0.145 

0.352 

0.285 

0.451 

0.212 

0.409 

0.225 

0.418 

33 

ACQ  CAREER  FLD:  Facilities  Engineering 

0.113 

0.317 

0.036 

0.188 

0.078 

0.267 

0.061 

0.239 

34 

ACQ  CAREER  FLD:  Information  Tech 

0.046 

0.209 

0.034 

0.181 

0.04 

0.197 

0.043 

0.203 

35 

ACQ  CAREER  FLD:  Life  Cycle  Logistics 

0.159 

0.365 

0.097 

0.296 

0.126 

0.332 

0.11 

0.313 

36 

ACQ  CAREER  FLD:  Other 

0.041 

0.197 

0.02 

0.142 

0.032 

0.175 

0.036 

0.187 

37 

ACQ  CAREER  FLD:  Prod,  Quality  &  Manuf 

0.09 

0.286 

0.057 

0.231 

0.072 

0.259 

0.07 

0.256 

38 

ACQ  CAREER  FLD:  Program  Management 

0.078 

0.268 

0.07 

0.255 

0.075 

0.264 

0.072 

0.259 

39 

ACQ  CAREER  FLD:  Test  and  Eval 

0.029 

0.167 

0.057 

0.231 

0.045 

0.207 

0.045 

0.207 
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40 

41 

42 

43 

44 

45 

46 

47 

48 

49 

50 

51 

52 

53 

54 

55 

56 

57 

58 

59 

60 

61 


Balance  Factors 

Unweighted 

Internal  Hire  External  Hire 

Mean  SD  Mean  SD 

Internal  Hire 

Mean  SD 

Weighted 

External  Hire 

Mean  SD 

ACQ  CAREER  FLD:  Unknown/Missing 

0.082 

0.274 

0.079 

0.27 

0.076 

0.264 

0.08 

0.272 

RETIREMENT  PLAN:  CSRS 

0.198 

0.398 

0.024 

0.154 

0.116 

0.321 

0.107 

0.309 

RETIREMENT  PLAN:  FERS 

0.8 

0.4 

0.962 

0.192 

0.876 

0.33 

0.884 

0.321 

RETIREMENT  PLAN:  Other 

0.002 

0.048 

0.014 

0.117 

0.008 

0.089 

0.009 

0.096 

CAREER  LEVEL:  Entry-Level 

0.212 

0.409 

0.407 

0.491 

0.3 

0.458 

0.328 

0.47 

CAREER  LEVEL:  Mid-Level 

0.699 

0.459 

0.512 

0.5 

0.613 

0.487 

0.589 

0.492 

CAREER  LEVEL:  Sen/Exec 

0.083 

0.275 

0.065 

0.247 

0.077 

0.266 

0.072 

0.258 

CAREER  LEVEL:  Uncategorized 

0.006 

0.074 

0.015 

0.122 

0.011 

0.103 

0.011 

0.106 

COHORT  YEAR:  2004 

0.402 

0.49 

0.316 

0.465 

0.368 

0.482 

0.352 

0.477 

COHORT  YEAR:  2005 

0.363 

0.481 

0.333 

0.471 

0.344 

0.475 

0.348 

0.476 

COHORT  YEAR:  2007 

0.234 

0.424 

0.35 

0.477 

0.289 

0.453 

0.3 

0.458 

YRS  TO  RET  ELIG:  <5  Yrs  to  Go 

0.114 

0.318 

0.037 

0.188 

0.077 

0.267 

0.071 

0.257 

YRS  TO  RET  ELIG:  10  to  19  Yrs  to  Go 

0.309 

0.462 

0.316 

0.465 

0.317 

0.465 

0.318 

0.466 

YRS  TO  RET  ELIG:  20+  YRs  to  Go 

0.341 

0.474 

0.565 

0.496 

0.438 

0.496 

0.464 

0.499 

YRS  TO  RET  ELIG:  5-9  Yrs  to  Go 

0.165 

0.371 

0.073 

0.261 

0.125 

0.33 

0.115 

0.318 

YRS  TO  RET  ELIG:  Ret  Eligible 

0.071 

0.257 

0.009 

0.095 

0.043 

0.202 

0.032 

0.177 

PAY  PLAN:  Broad  GS 

0.838 

0.368 

0.719 

0.449 

0.781 

0.413 

0.77 

0.421 

PAY  PLAN:  Demo 

0.119 

0.323 

0.217 

0.412 

0.163 

0.369 

0.177 

0.381 

PAY  PLAN:  NSPS 

0.038 

0.19 

0.05 

0.217 

0.046 

0.209 

0.043 

0.204 

PAY  PLAN:  Other 

0.005 

0.073 

0.014 

0.118 

0.01 

0.099 

0.01 

0.1 

HCAP:  Handicapped 

0.084 

0.277 

0.073 

0.26 

0.081 

0.273 

0.081 

0.272 

HCAP:  Not  Handicapped 

0.916 

0.277 

0.927 

0.26 

0.919 

0.273 

0.919 

0.272 
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Figure  D.2.2.  After  Weighting,  Several  Differences  in  Means  are  no  Longer  Statistically  Significant 
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D.3  Combination  Pre-Policy  Cohort  Group 


Table  D.3.1.  Means  and  Standard  Deviations  Before  and  After  Propensity-Score  Weighting 


Unweighted  Weighted 

Balance  Factors  Internal  Hire  External  Hire  Internal  Hire  External  Hire 

Mean  SD  Mean  SD  Mean  SD  Mean  SD 


AGE 

42.491 

10.84 

36.909 

1 1 .006 

40.039 

1 1 .253 

39.563 

11.219 

LN(COMPENSATION) 

10.853 

0.314 

10.78 

0.358 

10.825 

0.334 

10.808 

0.342 

RACE:  Other 

0.24 

0.427 

0.212 

0.409 

0.225 

0.418 

0.221 

0.415 

RACE:  White 

0.76 

0.427 

0.788 

0.409 

0.775 

0.418 

0.779 

0.415 

EDUCATION:  BA 

0.386 

0.487 

0.545 

0.498 

0.456 

0.498 

0.468 

0.499 

EDUCATION:  Less  than  BA 

0.432 

0.495 

0.179 

0.383 

0.319 

0.466 

0.3 

0.458 

EDUCATION:  More  than  BA 

0.182 

0.386 

0.276 

0.447 

0.225 

0.418 

0.232 

0.422 

DOD  ORG:  Air  Force 

0.154 

0.361 

0.148 

0.356 

0.157 

0.364 

0.152 

0.359 

DOD  ORG:  Army 

0.497 

0.5 

0.361 

0.48 

0.438 

0.496 

0.414 

0.493 

DOD  ORG:  Navy 

0.225 

0.417 

0.374 

0.484 

0.284 

0.451 

0.307 

0.461 

DOD  ORG:  Other 

0.124 

0.33 

0.117 

0.322 

0.122 

0.327 

0.127 

0.332 

REGION:  East  North  Central 

0.133 

0.34 

0.097 

0.295 

0.116 

0.32 

0.11 

0.313 

REGION:  East  South  Central 

0.096 

0.294 

0.076 

0.265 

0.084 

0.277 

0.073 

0.26 

REGION:  Mid  Atlantic 

0.106 

0.308 

0.111 

0.315 

0.107 

0.309 

0.109 

0.312 

REGION:  Mountain 

0.067 

0.25 

0.064 

0.245 

0.067 

0.25 

0.07 

0.255 

REGION:  New  England 

0.033 

0.178 

0.042 

0.2 

0.035 

0.183 

0.039 

0.193 

REGION:  Other  or  Missing 

0.035 

0.185 

0.018 

0.133 

0.03 

0.17 

0.029 

0.169 

REGION:  Pacific 

0.119 

0.324 

0.148 

0.355 

0.13 

0.337 

0.141 

0.348 

REGION:  South  Atlantic 

0.307 

0.461 

0.384 

0.486 

0.344 

0.475 

0.348 

0.476 

REGION:  West  North  Central 

0.018 

0.133 

0.012 

0.107 

0.016 

0.125 

0.015 

0.121 

REGION:  West  South  Central 

0.086 

0.28 

0.049 

0.216 

0.072 

0.258 

0.065 

0.247 

GENDER:  Female 

0.432 

0.495 

0.278 

0.448 

0.362 

0.481 

0.352 

0.478 

GENDER:  Male 

0.568 

0.495 

0.722 

0.448 

0.638 

0.481 

0.648 

0.478 

POSITION  TYPE:  Administrative 

0.516 

0.5 

0.265 

0.442 

0.405 

0.491 

0.368 

0.482 

POSITION  TYPE:  Other 

0.014 

0.116 

0.012 

0.107 

0.013 

0.115 

0.014 

0.117 

POSITION  TYPE:  Professional 

0.398 

0.49 

0.69 

0.463 

0.527 

0.499 

0.559 

0.497 

POSITION  TYPE:  Technical 

0.072 

0.259 

0.033 

0.18 

0.055 

0.228 

0.059 

0.236 

SUPERVISOR:  All  Other  Positions 

0.887 

0.316 

0.952 

0.213 

0.915 

0.279 

0.925 

0.264 

SUPERVISOR:  Supvisor/Mngr/Ldr 

0.113 

0.316 

0.048 

0.213 

0.085 

0.279 

0.075 

0.264 

ACQ  CAREER  FLD:  Business  (CE  or  FM) 

0.116 

0.32 

0.045 

0.207 

0.085 

0.278 

0.085 

0.28 

ACQ  CAREER  FLD:  Contracting 

0.121 

0.326 

0.201 

0.401 

0.162 

0.368 

0.169 

0.375 

ACQ  CAREER  FLD:  Engineering 

0.173 

0.378 

0.335 

0.472 

0.243 

0.429 

0.26 

0.439 

ACQ  CAREER  FLD:  Facilities  Engineering 

0.066 

0.248 

0.036 

0.188 

0.053 

0.224 

0.045 

0.208 

ACQ  CAREER  FLD:  Information  Tech 

0.072 

0.259 

0.036 

0.187 

0.057 

0.232 

0.051 

0.219 

ACQ  CAREER  FLD:  Life  Cycle  Logistics 

0.139 

0.346 

0.084 

0.277 

0.111 

0.314 

0.1 

0.3 

ACQ  CAREER  FLD:  Other 

0.046 

0.209 

0.022 

0.146 

0.035 

0.184 

0.044 

0.205 

ACQ  CAREER  FLD:  Prod,  Quality  &  Manuf 

0.072 

0.259 

0.044 

0.206 

0.058 

0.234 

0.057 

0.232 

ACQ  CAREER  FLD:  Program  Management 

0.092 

0.289 

0.068 

0.251 

0.082 

0.275 

0.071 

0.256 

ACQ  CAREER  FLD:  Test  and  Eval 

0.029 

0.167 

0.066 

0.248 

0.046 

0.209 

0.049 

0.216 

ill 


40 

41 

42 

43 

44 

45 

46 

47 

48 

49 

50 

51 

52 

53 

54 

55 

56 

57 

58 

59 

60 

61 

62 

63 

64 


Balance  Factors 

Unweighted 

Internal  Hire  External  Hire 

Mean  SD  Mean  SD 

Internal  Hire 

Mean  SD 

Weighted 

External  Hire 

Mean  SD 

ACQ  CAREER  FLD:  Unknown/Missing 

0.074 

0.262 

0.063 

0.244 

0.068 

0.251 

0.068 

0.252 

RETIREMENT  PLAN:  CSRS 

0.265 

0.442 

0.036 

0.185 

0.164 

0.37 

0.142 

0.349 

RETIREMENT  PLAN:  FERS 

0.733 

0.443 

0.952 

0.213 

0.829 

0.376 

0.85 

0.357 

RETIREMENT  PLAN:  Other 

0.002 

0.046 

0.012 

0.11 

0.007 

0.084 

0.008 

0.088 

CAREER  LEVEL:  Entry-Level 

0.19 

0.392 

0.394 

0.489 

0.273 

0.446 

0.307 

0.461 

CAREER  LEVEL:  Mid-Level 

0.722 

0.448 

0.537 

0.499 

0.646 

0.478 

0.619 

0.486 

CAREER  LEVEL:  Sen/Exec 

0.083 

0.276 

0.054 

0.226 

0.071 

0.257 

0.063 

0.244 

CAREER  LEVEL:  Uncategorized 

0.005 

0.071 

0.015 

0.121 

0.01 

0.098 

0.011 

0.103 

COHORT  YEAR:  2001 

0.181 

0.385 

0.117 

0.322 

0.149 

0.356 

0.133 

0.34 

COHORT  YEAR:  2002 

0.192 

0.394 

0.173 

0.378 

0.177 

0.382 

0.178 

0.383 

COHORT  YEAR:  2003 

0.139 

0.346 

0.171 

0.376 

0.147 

0.354 

0.16 

0.367 

COHORT  YEAR:  2004 

0.196 

0.397 

0.17 

0.376 

0.194 

0.395 

0.188 

0.391 

COHORT  YEAR:  2005 

0.177 

0.382 

0.18 

0.384 

0.179 

0.384 

0.185 

0.388 

COHORT  YEAR:  2007 

0.114 

0.318 

0.189 

0.391 

0.154 

0.361 

0.156 

0.363 

YRS  TO  RET  ELIG:  <5  Yrs  to  Go 

0.117 

0.321 

0.037 

0.188 

0.081 

0.273 

0.073 

0.259 

YRS  TO  RET  ELIG:  10  to  19  Yrs  to  Go 

0.332 

0.471 

0.316 

0.465 

0.331 

0.471 

0.327 

0.469 

YRS  TO  RET  ELIG:  20+  YRs  to  Go 

0.295 

0.456 

0.562 

0.496 

0.404 

0.491 

0.438 

0.496 

YRS  TO  RET  ELIG:  5-9  Yrs  to  Go 

0.184 

0.387 

0.075 

0.264 

0.138 

0.344 

0.126 

0.332 

YRS  TO  RET  ELIG:  Ret  Eligible 

0.072 

0.259 

0.01 

0.1 

0.045 

0.208 

0.036 

0.187 

PAY  PLAN:  Broad  GS 

0.855 

0.352 

0.72 

0.449 

0.795 

0.404 

0.782 

0.413 

PAY  PLAN:  Demo 

0.122 

0.328 

0.239 

0.427 

0.171 

0.377 

0.186 

0.389 

PAY  PLAN:  NSPS 

0.018 

0.134 

0.027 

0.161 

0.025 

0.156 

0.022 

0.148 

PAY  PLAN:  Other 

0.004 

0.066 

0.014 

0.117 

0.009 

0.093 

0.01 

0.099 

HCAP:  Handicapped 

0.084 

0.278 

0.081 

0.273 

0.085 

0.279 

0.087 

0.282 

HCAP:  Not  Handicapped 

0.916 

0.278 

0.919 

0.273 

0.915 

0.279 

0.913 

0.282 
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Figure  D.3.1.  Absolute  Standard  Differences  in  Means  Decrease  Substantially  after  Weighting 


Figure  D.3.2.  After  Weighting,  One  or  Two  Differences  in  Means  are  no  Longer  Statistically 

Significant 
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Appendix  E.  Balance  Results  for  DoD-Experienced  External  Hires 
vs.  Non-DoD-Experienced  External  Hires 


Sections  E.  1-E.3  show  how  the  pre-policy  cohort  groups  analyzed  in  Part  I  of  the  retention 

Q7 

analysis  (see  Table  E.l,  below)  balance,  in  terms  of  the  observable  characteristics  listed  in 
Table  6.2,  before  and  after  propensity-score  weight  implementation.  Tables  E.  1 . 1-E.3. 1  list  the 
variable  means  and  standard  deviations  before  and  after  weighting.  And  the  figures  illustrate  how 
effectively  the  propensity-score  weights  bring  the  variable  means  of  the  two  groups  (external 
hires  with  prior  DoD  experience  vs.  those  with  no  prior  DoD  experience)  into  balance. 

For  all  three  cohort  groups  analyzed,  the  applied  propensity-score  weights  significantly 
improve  the  balance  between  the  two  groups.  Figures  E.l.l-E.3.1  illustrate  that  the  differences  in 
the  variable  means  between  the  two  groups  decrease  substantially  after  weighting.  In  no  cases — 
post-weighting — is  the  “Absolute  standard  difference”  in  variable  means  between  groups 
greater  than  0.2,  which  is  the  threshold  referenced  in  the  RAND  TWANG  (Griffin  et  ah,  2014; 
Griffin  and  McCaffrey,  2015d;  Griffin  and  McCaffrey,  2015c).99 

Figures  E.1.2-E.3.2  show  that  after  applying  propensity-score  weights  some  of  the 
differences  in  variable  means  between  groups  are  no  longer  statistically  significant. 

The  tables  and  figures  presented  in  sections  E.1-E.3  come  from  the  output  of  RAND’s 
TWANG  (Griffin  et  ah,  2014). 

Table  E.l.  Pre-Policy  Cohort  Groups  Analyzed  in  Retention  Analysis  Part  I 

Cohort  Group  Name  FY  Cohorts  Included 

Early  Pre-policy  cohort  group  2001-2003 

Later  pre-policy  cohort  group  2004,  2005,  and  2007 

Combination  pre-policy  cohort  group  2001-2005  and  2007 


97 

Griffin  and  McCaffrey  (2015b)  and  Griffin  and  McCaffrey  (2015a)  state  that  balance  assessments  should  consider 
differences  in  variable  means  and  variable  distributions  between  groups.  The  balance  analyses  in  this  study  focus  on 
comparing  variable  means  and  do  not  include  comparing  variable  distributions  because  most  of  the  variables  used 
are  categorical — for  these  variables  the  means  and  distributions  are  essentially  the  same  thing. 

98 

This  phrase  stems  from  Griffin  and  McCaffrey  (2015c)  (p.  28).  The  phrase  is  used  throughout  the  following 
section. 

99 

Some  researchers  use  a  stricter  threshold  of  0. 1  (Austin  and  Stuart,  2015). 
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E.1  Early  Pre-Policy  Cohort  Group 


Table  E.1.1.  Means  and  Standard  Deviations  Before  and  After  Propensity-Score  Weighting 


Unweighted  Weighted 


Balance  Factors 

Mean 

DoD 

SD 

Mean 

No  DoD 

SD 

DoD 

Mean  SD 

Mean 

No  DoD 

SD 

1 

AGE 

42.888 

8.379 

31.77 

10.251 

37.638 

10.669 

36.082 

10.909 

2 

LN(COMPENSATION) 

10.859 

0.345 

10.677 

0.314 

10.78 

0.34 

10.743 

0.336 

3 

RACE:  Other 

0.192 

0.394 

0.235 

0.424 

0.209 

0.407 

0.217 

0.412 

4 

RACE:  White 

0.808 

0.394 

0.765 

0.424 

0.791 

0.407 

0.783 

0.412 

5 

EDUCATION:  BA 

0.4 

0.49 

0.691 

0.462 

0.541 

0.498 

0.583 

0.493 

6 

EDUCATION:  Less  than  BA 

0.28 

0.449 

0.079 

0.27 

0.189 

0.392 

0.156 

0.363 

7 

EDUCATION:  More  than  BA 

0.32 

0.467 

0.23 

0.421 

0.27 

0.444 

0.262 

0.44 

8 

DOD  ORG:  Air  Force 

0.204 

0.403 

0.12 

0.325 

0.17 

0.375 

0.145 

0.352 

9 

DOD  ORG:  Army 

0.312 

0.463 

0.318 

0.466 

0.317 

0.465 

0.33 

0.47 

10 

DOD  ORG:  Navy 

0.384 

0.486 

0.497 

0.5 

0.431 

0.495 

0.449 

0.497 

11 

DOD  ORG:  Other 

0.1 

0.3 

0.065 

0.246 

0.082 

0.274 

0.075 

0.264 

12 

REGION:  East  North  Central 

0.084 

0.278 

0.084 

0.277 

0.083 

0.277 

0.081 

0.273 

13 

REGION:  East  South  Central 

0.067 

0.25 

0.071 

0.257 

0.065 

0.246 

0.069 

0.253 

14 

REGION:  Mid  Atlantic 

0.059 

0.236 

0.116 

0.321 

0.1 

0.3 

0.093 

0.291 

15 

REGION:  Mountain 

0.088 

0.283 

0.045 

0.207 

0.063 

0.243 

0.063 

0.244 

16 

REGION:  New  England 

0.044 

0.204 

0.051 

0.22 

0.047 

0.211 

0.049 

0.215 

17 

REGION:  Other  or  Missing 

0.028 

0.166 

0.007 

0.082 

0.018 

0.133 

0.014 

0.118 

18 

REGION:  Pacific 

0.147 

0.354 

0.198 

0.399 

0.161 

0.367 

0.187 

0.39 

19 

REGION:  South  Atlantic 

0.411 

0.492 

0.378 

0.485 

0.402 

0.49 

0.387 

0.487 

20 

REGION:  West  North  Central 

0.013 

0.115 

0.007 

0.085 

0.01 

0.097 

0.009 

0.092 

21 

REGION:  West  South  Central 

0.059 

0.236 

0.043 

0.203 

0.052 

0.222 

0.049 

0.216 

22 

GENDER:  Female 

0.199 

0.399 

0.311 

0.463 

0.26 

0.439 

0.271 

0.444 

23 

GENDER:  Male 

0.801 

0.399 

0.689 

0.463 

0.74 

0.439 

0.729 

0.444 

24 

POSITION  TYPE:  Administrative 

0.386 

0.487 

0.107 

0.309 

0.25 

0.433 

0.207 

0.405 

25 

POSITION  TYPE:  Other 

0.016 

0.125 

0.01 

0.099 

0.015 

0.123 

0.011 

0.106 

26 

POSITION  TYPE:  Professional 

0.554 

0.497 

0.868 

0.338 

0.703 

0.457 

0.758 

0.429 

27 

POSITION  TYPE:  Technical 

0.044 

0.206 

0.015 

0.122 

0.031 

0.173 

0.024 

0.153 

28 

SUPERVISOR:  All  Other  Positions 

0.923 

0.267 

0.99 

0.1 

0.955 

0.208 

0.974 

0.158 

29 

SUPERVISOR:  Supvisor/Mngr/Ldr 

0.077 

0.267 

0.01 

0.1 

0.045 

0.208 

0.026 

0.158 

30 

ACQ  CAREER  FLD:  Business  (CE  or  FM) 

0.044 

0.206 

0.034 

0.182 

0.039 

0.193 

0.04 

0.196 

31 

ACQ  CAREER  FLD:  Contracting 

0.208 

0.406 

0.162 

0.369 

0.189 

0.392 

0.179 

0.383 

32 

ACQ  CAREER  FLD:  Engineering 

0.256 

0.436 

0.518 

0.5 

0.384 

0.486 

0.421 

0.494 

33 

ACQ  CAREER  FLD:  Facilities  Engineering 

0.026 

0.159 

0.046 

0.209 

0.042 

0.2 

0.041 

0.198 

34 

ACQ  CAREER  FLD:  Information  Tech 

0.056 

0.229 

0.025 

0.157 

0.04 

0.196 

0.038 

0.192 

35 

ACQ  CAREER  FLD:  Life  Cycle  Logistics 

0.116 

0.32 

0.025 

0.157 

0.068 

0.252 

0.058 

0.233 

36 

ACQ  CAREER  FLD:  Other 

0.032 

0.177 

0.015 

0.121 

0.025 

0.157 

0.023 

0.148 

37 

ACQ  CAREER  FLD:  Prod,  Quality  &  Manuf 

0.042 

0.2 

0.02 

0.139 

0.03 

0.171 

0.026 

0.158 

38 

ACQ  CAREER  FLD:  Program  Management 

0.107 

0.308 

0.027 

0.163 

0.074 

0.261 

0.054 

0.226 

39 

ACQ  CAREER  FLD:  Test  and  Eval 

0.061 

0.239 

0.09 

0.286 

0.064 

0.244 

0.077 

0.266 
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40 

41 

42 

43 

44 

45 

46 

47 

48 

49 

50 

51 

52 

53 

54 

55 

56 

57 

58 

59 

60 


Balance  Factors 

Mean 

Unweighted 

DoD  No  DoD 

SD  Mean  SD 

DoD 

Mean  SD 

Weighted 

No  DoD 

Mean  SD 

ACQ  CAREER  FLD:  Unknown/Missing 

0.054 

0.225 

0.037 

0.189 

0.045 

0.208 

0.045 

0.207 

RETIREMENT  PLAN:  CSRS 

0.094 

0.292 

0.008 

0.088 

0.053 

0.223 

0.029 

0.168 

RETIREMENT  PLAN:  FERS 

0.89 

0.313 

0.987 

0.112 

0.937 

0.243 

0.962 

0.19 

RETIREMENT  PLAN:  Other 

0.016 

0.126 

0.005 

0.071 

0.01 

0.1 

0.008 

0.092 

CAREER  LEVEL:  Entry-Level 

0.183 

0.387 

0.554 

0.497 

0.344 

0.475 

0.411 

0.492 

CAREER  LEVEL:  Mid-Level 

0.722 

0.448 

0.426 

0.494 

0.597 

0.49 

0.551 

0.497 

CAREER  LEVEL:  Sen/Exec 

0.071 

0.257 

0.014 

0.115 

0.043 

0.204 

0.028 

0.165 

CAREER  LEVEL:  Uncategorized 

0.023 

0.151 

0.007 

0.081 

0.015 

0.121 

0.01 

0.099 

COHORT  YEAR:  2001 

0.259 

0.438 

0.25 

0.433 

0.235 

0.424 

0.258 

0.437 

COHORT  YEAR:  2002 

0.356 

0.479 

0.393 

0.488 

0.379 

0.485 

0.388 

0.487 

COHORT  YEAR:  2003 

0.385 

0.487 

0.357 

0.479 

0.386 

0.487 

0.354 

0.478 

YRS  TO  RET  ELIG:  <5  Yrs  to  Go 

0.05 

0.218 

0.025 

0.156 

0.039 

0.193 

0.032 

0.175 

YRS  TO  RET  ELIG:  10  to  19  Yrs  to  Go 

0.512 

0.5 

0.143 

0.35 

0.333 

0.471 

0.28 

0.449 

YRS  TO  RET  ELIG:  20+  YRs  to  Go 

0.296 

0.456 

0.792 

0.406 

0.534 

0.499 

0.61 

0.488 

YRS  TO  RET  ELIG:  5-9  Yrs  to  Go 

0.123 

0.328 

0.036 

0.187 

0.082 

0.274 

0.072 

0.259 

YRS  TO  RET  ELIG:  Ret  Eligible 

0.019 

0.136 

0.004 

0.064 

0.012 

0.109 

0.007 

0.08 

PAY  PLAN:  Broad  GS 

0.765 

0.424 

0.682 

0.466 

0.714 

0.452 

0.714 

0.452 

PAY  PLAN:  Demo 

0.212 

0.408 

0.313 

0.464 

0.271 

0.444 

0.277 

0.448 

PAY  PLAN:  Other 

0.023 

0.151 

0.005 

0.071 

0.015 

0.121 

0.009 

0.092 

HCAP:  Handicapped 

0.116 

0.32 

0.067 

0.25 

0.092 

0.289 

0.082 

0.275 

HCAP:  Not  Handicapped 

0.884 

0.32 

0.933 

0.25 

0.908 

0.289 

0.918 

0.275 
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Figure  E.1.1.  Absolute  Standard  Differences  in  Means  Decrease  Substantially  After  Weighting 


Figure  E.1.2.  After  Weighting,  Some  Differences  in  Means  are  no  Longer  Statistically  Significant 


Rank  of  p-value  rank  for  pretreatment  variables 
(hollow  is  weighted,  solid  is  unweighted) 
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E.2  Later  Pre-Policy  Cohort  Group 


Table  E.2.1.  Means  and  Standard  Deviations  Before  and  After  Propensity-Score  Weighting 


Unweighted  Weighted 


Balance  Factors 

Mean 

DoD 

SD 

Mean 

No  DoD 

SD 

DoD 

Mean  SD 

Mean 

No  DoD 

SD 

1 

AGE 

43.306 

8.367 

31.223 

10.017 

37.545 

10.852 

35.875 

10.998 

2 

LN(COMPENSATION) 

10.946 

0.363 

10.665 

0.325 

10.814 

0.37 

10.775 

0.363 

3 

RACE:  Other 

0.206 

0.405 

0.212 

0.409 

0.226 

0.418 

0.203 

0.402 

4 

RACE:  White 

0.794 

0.405 

0.788 

0.409 

0.774 

0.418 

0.797 

0.402 

5 

EDUCATION:  BA 

0.371 

0.483 

0.68 

0.466 

0.529 

0.499 

0.573 

0.495 

6 

EDUCATION:  Less  than  BA 

0.3 

0.458 

0.082 

0.275 

0.196 

0.397 

0.16 

0.367 

7 

EDUCATION:  More  than  BA 

0.329 

0.47 

0.238 

0.426 

0.275 

0.446 

0.267 

0.442 

8 

DOD  ORG:  Air  Force 

0.197 

0.398 

0.089 

0.284 

0.143 

0.35 

0.121 

0.326 

9 

DOD  ORG:  Army 

0.387 

0.487 

0.411 

0.492 

0.385 

0.487 

0.401 

0.49 

10 

DOD  ORG:  Navy 

0.282 

0.45 

0.341 

0.474 

0.329 

0.47 

0.327 

0.469 

11 

DOD  ORG:  Other 

0.134 

0.34 

0.16 

0.366 

0.143 

0.35 

0.151 

0.358 

12 

REGION:  East  North  Central 

0.102 

0.303 

0.112 

0.315 

0.103 

0.304 

0.106 

0.308 

13 

REGION:  East  South  Central 

0.08 

0.272 

0.083 

0.276 

0.082 

0.275 

0.075 

0.263 

14 

REGION:  Mid  Atlantic 

0.064 

0.245 

0.187 

0.39 

0.111 

0.314 

0.138 

0.345 

15 

REGION:  Mountain 

0.081 

0.273 

0.048 

0.213 

0.062 

0.241 

0.059 

0.235 

16 

REGION:  New  England 

0.036 

0.186 

0.037 

0.189 

0.04 

0.195 

0.038 

0.192 

17 

REGION:  Other  or  Missing 

0.031 

0.174 

0.008 

0.09 

0.021 

0.145 

0.017 

0.13 

18 

REGION:  Pacific 

0.122 

0.327 

0.128 

0.334 

0.12 

0.325 

0.123 

0.329 

19 

REGION:  South  Atlantic 

0.398 

0.49 

0.357 

0.479 

0.399 

0.49 

0.394 

0.489 

20 

REGION:  West  North  Central 

0.017 

0.129 

0.009 

0.095 

0.012 

0.109 

0.011 

0.106 

21 

REGION:  West  South  Central 

0.068 

0.252 

0.031 

0.173 

0.05 

0.219 

0.038 

0.19 

22 

GENDER:  Female 

0.211 

0.408 

0.366 

0.482 

0.31 

0.463 

0.318 

0.466 

23 

GENDER:  Male 

0.789 

0.408 

0.634 

0.482 

0.69 

0.463 

0.682 

0.466 

24 

POSITION  TYPE:  Administrative 

0.444 

0.497 

0.154 

0.361 

0.297 

0.457 

0.25 

0.433 

25 

POSITION  TYPE:  Other 

0.013 

0.112 

0.009 

0.092 

0.01 

0.099 

0.011 

0.105 

26 

POSITION  TYPE:  Professional 

0.487 

0.5 

0.816 

0.387 

0.655 

0.475 

0.705 

0.456 

27 

POSITION  TYPE:  Technical 

0.056 

0.231 

0.021 

0.144 

0.039 

0.193 

0.034 

0.181 

28 

SUPERVISOR:  All  Other  Positions 

0.898 

0.302 

0.99 

0.102 

0.943 

0.233 

0.957 

0.202 

29 

SUPERVISOR:  Supvisor/Mngr/Ldr 

0.102 

0.302 

0.01 

0.102 

0.057 

0.233 

0.043 

0.202 

30 

ACQ  CAREER  FLD:  Business  (CE  or  FM) 

0.05 

0.219 

0.049 

0.217 

0.055 

0.228 

0.052 

0.221 

31 

ACQ  CAREER  FLD:  Contracting 

0.236 

0.425 

0.198 

0.398 

0.224 

0.417 

0.222 

0.416 

32 

ACQ  CAREER  FLD:  Engineering 

0.165 

0.371 

0.388 

0.487 

0.277 

0.447 

0.303 

0.459 

33 

ACQ  CAREER  FLD:  Facilities  Engineering 

0.027 

0.162 

0.045 

0.207 

0.041 

0.199 

0.039 

0.193 

34 

ACQ  CAREER  FLD:  Information  Tech 

0.044 

0.206 

0.025 

0.155 

0.033 

0.179 

0.036 

0.187 

35 

ACQ  CAREER  FLD:  Life  Cycle  Logistics 

0.155 

0.362 

0.047 

0.211 

0.098 

0.297 

0.078 

0.268 

36 

ACQ  CAREER  FLD:  Other 

0.029 

0.169 

0.013 

0.112 

0.021 

0.145 

0.021 

0.143 

37 

ACQ  CAREER  FLD:  Prod,  Quality  &  Manuf 

0.076 

0.265 

0.04 

0.196 

0.054 

0.226 

0.049 

0.216 

38 

ACQ  CAREER  FLD:  Program  Management 

0.12 

0.325 

0.027 

0.161 

0.072 

0.258 

0.06 

0.238 

39 

ACQ  CAREER  FLD:  Test  and  Eval 

0.05 

0.217 

0.063 

0.242 

0.051 

0.219 

0.058 

0.234 
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40 

41 

42 

43 

44 

45 

46 

47 

48 

49 

50 

51 

52 

53 

54 

55 

56 

57 

58 

59 

60 

61 


Balance  Factors 

Mean 

Unweighted 

DoD  No  DoD 

SD  Mean  SD 

DoD 

Mean  SD 

Weighted 

No  DoD 

Mean  SD 

ACQ  CAREER  FLD:  Unknown/Missing 

0.048 

0.213 

0.106 

0.308 

0.074 

0.262 

0.082 

0.275 

RETIREMENT  PLAN:  CSRS 

0.048 

0.214 

0.004 

0.063 

0.027 

0.161 

0.015 

0.121 

RETIREMENT  PLAN:  FERS 

0.932 

0.252 

0.987 

0.112 

0.96 

0.195 

0.972 

0.164 

RETIREMENT  PLAN:  Other 

0.02 

0.141 

0.009 

0.093 

0.013 

0.114 

0.013 

0.113 

CAREER  LEVEL:  Entry-Level 

0.184 

0.387 

0.6 

0.49 

0.383 

0.486 

0.435 

0.496 

CAREER  LEVEL:  Mid-Level 

0.675 

0.469 

0.372 

0.483 

0.532 

0.499 

0.498 

0.5 

CAREER  LEVEL:  Sen/Exec 

0.122 

0.328 

0.016 

0.126 

0.07 

0.256 

0.054 

0.226 

CAREER  LEVEL:  Uncategorized 

0.019 

0.137 

0.011 

0.107 

0.014 

0.118 

0.012 

0.11 

COHORT  YEAR:  2004 

0.301 

0.459 

0.329 

0.47 

0.322 

0.467 

0.334 

0.472 

COHORT  YEAR:  2005 

0.348 

0.476 

0.321 

0.467 

0.346 

0.476 

0.323 

0.468 

COHORT  YEAR:  2007 

0.351 

0.477 

0.35 

0.477 

0.332 

0.471 

0.343 

0.475 

YRS  TO  RET  ELIG:  <5  Yrs  to  Go 

0.052 

0.221 

0.024 

0.152 

0.038 

0.191 

0.034 

0.182 

YRS  TO  RET  ELIG:  10  to  19  Yrs  to  Go 

0.53 

0.499 

0.132 

0.338 

0.336 

0.472 

0.279 

0.449 

YRS  TO  RET  ELIG:  20+  YRs  to  Go 

0.283 

0.451 

0.808 

0.394 

0.537 

0.499 

0.614 

0.487 

YRS  TO  RET  ELIG:  5-9  Yrs  to  Go 

0.119 

0.324 

0.034 

0.18 

0.079 

0.27 

0.066 

0.248 

YRS  TO  RET  ELIG:  Ret  Eligible 

0.016 

0.124 

0.004 

0.06 

0.01 

0.099 

0.007 

0.082 

PAY  PLAN:  Broad  GS 

0.723 

0.448 

0.716 

0.451 

0.713 

0.452 

0.722 

0.448 

PAY  PLAN:  Demo 

0.186 

0.389 

0.244 

0.429 

0.224 

0.417 

0.223 

0.417 

PAY  PLAN:  NSPS 

0.072 

0.259 

0.03 

0.171 

0.049 

0.216 

0.043 

0.202 

PAY  PLAN:  Other 

0.019 

0.135 

0.01 

0.1 

0.014 

0.116 

0.012 

0.107 

HCAP:  Handicapped 

0.099 

0.298 

0.051 

0.22 

0.076 

0.265 

0.068 

0.253 

HCAP:  Not  Handicapped 

0.901 

0.298 

0.949 

0.22 

0.924 

0.265 

0.932 

0.253 
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Figure  E.2.1.  Absolute  Standard  Differences  in  Means  Decrease  Substantially  After  Weighting 


Figure  E.2.2.  After  Weighting,  Several  Differences  in  Means  are  no  Longer  Statistically  Significant 
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Rank  of  p-value  rank  for  pretreatment  variables 
(hollow  is  weighted,  solid  is  unweighted) 
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E.3  Combination  Pre-Policy  Cohort  Group 


Table  E.3.1.  Means  and  Standard  Deviations  Before  and  After  Propensity-Score  Weighting 


Balance  Factors 

Mean 

Unweighted 

DoD 

SD  Mean 

No  DoD 

SD 

Mean 

DoD 

SD 

Weighted 

Mean 

No  DoD 

SD 

AGE 

43.111 

8.375 

31.473 

10.128 

37.417 

10.844 

36.145 

10.984 

LN(COMPENSATION) 

10.905 

0.357 

10.67 

0.32 

10.798 

0.353 

10.765 

0.352 

RACE:  Other 

0.2 

0.4 

0.223 

0.416 

0.22 

0.414 

0.21 

0.408 

RACE:  White 

0.8 

0.4 

0.777 

0.416 

0.78 

0.414 

0.79 

0.408 

EDUCATION:  BA 

0.385 

0.486 

0.685 

0.465 

0.541 

0.498 

0.574 

0.495 

EDUCATION:  Less  than  BA 

0.291 

0.454 

0.081 

0.273 

0.187 

0.39 

0.161 

0.368 

EDUCATION:  More  than  BA 

0.325 

0.468 

0.234 

0.423 

0.272 

0.445 

0.265 

0.441 

DOD  ORG:  Air  Force 

0.2 

0.4 

0.103 

0.304 

0.152 

0.359 

0.134 

0.34 

DOD  ORG:  Army 

0.352 

0.478 

0.369 

0.482 

0.357 

0.479 

0.37 

0.483 

DOD  ORG:  Navy 

0.33 

0.47 

0.412 

0.492 

0.379 

0.485 

0.38 

0.485 

DOD  ORG:  Other 

0.118 

0.323 

0.116 

0.321 

0.112 

0.315 

0.116 

0.321 

REGION:  East  North  Central 

0.094 

0.292 

0.099 

0.299 

0.094 

0.292 

0.095 

0.294 

REGION:  East  South  Central 

0.074 

0.262 

0.078 

0.267 

0.076 

0.265 

0.072 

0.258 

REGION:  Mid  Atlantic 

0.062 

0.241 

0.155 

0.362 

0.106 

0.308 

0.116 

0.32 

REGION:  Mountain 

0.084 

0.278 

0.046 

0.21 

0.061 

0.24 

0.062 

0.242 

REGION:  New  England 

0.039 

0.195 

0.043 

0.204 

0.043 

0.202 

0.043 

0.202 

REGION:  Other  or  Missing 

0.03 

0.171 

0.008 

0.087 

0.019 

0.138 

0.018 

0.132 

REGION:  Pacific 

0.133 

0.34 

0.16 

0.367 

0.138 

0.345 

0.153 

0.36 

REGION:  South  Atlantic 

0.404 

0.491 

0.366 

0.482 

0.401 

0.49 

0.389 

0.488 

REGION:  West  North  Central 

0.015 

0.123 

0.008 

0.09 

0.011 

0.103 

0.01 

0.1 

REGION:  West  South  Central 

0.064 

0.245 

0.036 

0.187 

0.05 

0.218 

0.042 

0.201 

GENDER:  Female 

0.205 

0.404 

0.341 

0.474 

0.291 

0.454 

0.295 

0.456 

GENDER:  Male 

0.795 

0.404 

0.659 

0.474 

0.709 

0.454 

0.705 

0.456 

POSITION  TYPE:  Administrative 

0.417 

0.493 

0.132 

0.339 

0.27 

0.444 

0.235 

0.424 

POSITION  TYPE:  Other 

0.014 

0.119 

0.009 

0.095 

0.011 

0.103 

0.011 

0.102 

POSITION  TYPE:  Professional 

0.518 

0.5 

0.84 

0.367 

0.685 

0.465 

0.726 

0.446 

POSITION  TYPE:  Technical 

0.051 

0.219 

0.018 

0.134 

0.034 

0.182 

0.029 

0.167 

SUPERVISOR:  All  Other  Positions 

0.91 

0.286 

0.99 

0.101 

0.949 

0.219 

0.963 

0.188 

SUPERVISOR:  Supvisor/Mngr/Ldr 

0.09 

0.286 

0.01 

0.101 

0.051 

0.219 

0.037 

0.188 

ACQ  CAREER  FLD:  Business  (CE  or  FM) 

0.048 

0.213 

0.042 

0.202 

0.047 

0.213 

0.045 

0.208 

ACQ  CAREER  FLD:  Contracting 

0.223 

0.416 

0.181 

0.385 

0.205 

0.404 

0.205 

0.404 

ACQ  CAREER  FLD:  Engineering 

0.207 

0.405 

0.448 

0.497 

0.335 

0.472 

0.353 

0.478 

ACQ  CAREER  FLD:  Facilities  Engineering 

0.027 

0.161 

0.045 

0.208 

0.043 

0.202 

0.039 

0.194 

ACQ  CAREER  FLD:  Information  Tech 

0.05 

0.217 

0.025 

0.156 

0.035 

0.185 

0.038 

0.19 

ACQ  CAREER  FLD:  Life  Cycle  Logistics 

0.137 

0.344 

0.037 

0.189 

0.082 

0.274 

0.072 

0.258 

ACQ  CAREER  FLD:  Other 

0.031 

0.173 

0.014 

0.116 

0.022 

0.146 

0.021 

0.142 

ACQ  CAREER  FLD:  Prod,  Quality  &  Manuf 

0.06 

0.237 

0.031 

0.173 

0.042 

0.201 

0.038 

0.191 

ACQ  CAREER  FLD:  Program  Management 

0.114 

0.318 

0.027 

0.162 

0.072 

0.259 

0.059 

0.235 

ACQ  CAREER  FLD:  Test  and  Eval 

0.055 

0.227 

0.075 

0.264 

0.057 

0.231 

0.066 

0.249 
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41 

42 

43 

44 

45 

46 

47 

48 

49 

50 

51 

52 

53 

54 

55 

56 

57 

58 

59 

60 

61 

62 

63 

64 


Balance  Factors 

Mean 

Unweighted 

DoD  No  DoD 

SD  Mean  SD 

DoD 

Mean  SD 

Weighted 

No  DoD 

Mean  SD 

ACQ  CAREER  FLD:  Unknown/Missing 

0.05 

0.219 

0.075 

0.263 

0.059 

0.236 

0.064 

0.244 

RETIREMENT  PLAN:  CSRS 

0.07 

0.255 

0.006 

0.075 

0.038 

0.191 

0.028 

0.165 

RETIREMENT  PLAN:  FERS 

0.912 

0.283 

0.987 

0.112 

0.951 

0.217 

0.961 

0.195 

RETIREMENT  PLAN:  Other 

0.018 

0.134 

0.007 

0.083 

0.012 

0.107 

0.012 

0.107 

CAREER  LEVEL:  Entry-Level 

0.183 

0.387 

0.579 

0.494 

0.371 

0.483 

0.418 

0.493 

CAREER  LEVEL:  Mid-Level 

0.697 

0.46 

0.397 

0.489 

0.558 

0.497 

0.526 

0.499 

CAREER  LEVEL:  Sen/Exec 

0.099 

0.298 

0.015 

0.121 

0.057 

0.232 

0.045 

0.206 

CAREER  LEVEL:  Uncategorized 

0.021 

0.144 

0.009 

0.096 

0.014 

0.118 

0.011 

0.106 

COHORT  YEAR:  2001 

0.121 

0.326 

0.114 

0.318 

0.11 

0.313 

0.12 

0.325 

COHORT  YEAR:  2002 

0.166 

0.372 

0.18 

0.384 

0.176 

0.381 

0.178 

0.383 

COHORT  YEAR:  2003 

0.179 

0.384 

0.163 

0.37 

0.179 

0.384 

0.164 

0.37 

COHORT  YEAR:  2004 

0.161 

0.367 

0.179 

0.383 

0.172 

0.377 

0.179 

0.384 

COHORT  YEAR:  2005 

0.186 

0.389 

0.174 

0.379 

0.185 

0.388 

0.173 

0.379 

COHORT  YEAR:  2007 

0.187 

0.39 

0.19 

0.393 

0.178 

0.383 

0.185 

0.388 

YRS  TO  RET  ELIG:  <5  Yrs  to  Go 

0.051 

0.22 

0.024 

0.154 

0.038 

0.191 

0.033 

0.179 

YRS  TO  RET  ELIG:  10  to  19  Yrs  to  Go 

0.522 

0.5 

0.137 

0.343 

0.33 

0.47 

0.283 

0.451 

YRS  TO  RET  ELIG:  20+  YRs  to  Go 

0.289 

0.453 

0.8 

0.4 

0.543 

0.498 

0.601 

0.49 

YRS  TO  RET  ELIG:  5-9  Yrs  to  Go 

0.121 

0.326 

0.035 

0.184 

0.079 

0.27 

0.075 

0.263 

YRS  TO  RET  ELIG:  Ret  Eligible 

0.017 

0.13 

0.004 

0.062 

0.011 

0.104 

0.007 

0.085 

PAY  PLAN:  Broad  GS 

0.743 

0.437 

0.7 

0.458 

0.708 

0.455 

0.72 

0.449 

PAY  PLAN:  Demo 

0.198 

0.399 

0.275 

0.447 

0.252 

0.434 

0.247 

0.431 

PAY  PLAN:  NSPS 

0.038 

0.192 

0.016 

0.127 

0.026 

0.16 

0.022 

0.148 

PAY  PLAN:  Other 

0.021 

0.143 

0.008 

0.088 

0.014 

0.117 

0.01 

0.101 

HCAP:  Handicapped 

0.107 

0.309 

0.058 

0.235 

0.083 

0.275 

0.077 

0.267 

HCAP:  Not  Handicapped 

0.893 

0.309 

0.942 

0.235 

0.917 

0.275 

0.923 

0.267 
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Figure  E.3.1.  Absolute  Standard  Differences  in  Means  Decrease  Substantially  After  Weighting 


Figure  E.3.2.  After  Weighting,  Some  Differences  in  Means  are  no  Longer  Statistically  Significant 


Rank  of  p-value  rank  for  pretreatment  vanables 
(hollow  is  weighted,  solid  is  unweighted) 
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Appendix  F.  Retention  Analysis  Part  I:  Additional  Results 


Table  F.l.  Internal  Hires  Have  a  Lower  Hazard  of  Separating  From  the  Wider  DoD  Civilian 

Workforce  Compared  to  External  Hires 


Doubly  Robust  Cox  PH 

Pre-Policy  Cohort  Group 

Early 

Later 

Combination 

N 

27224 

28643 

55867 

Degrees  of  Freedom 

27223 

28642 

55866 

Estimation 

-0.086 

-0.140 

-0.107 

Hazard  Ratio 

0.918 

0.869 

0.899 

Standard  Error 

0.035 

0.030 

0.021 

p- value 

0.0141 

<.0001 

<.0001 
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Appendix  G.  The  Creation  of  the  Synthetic  Cohort  Groups 


This  analysis  uses  propensity  score  weights  (using  RAND’s  TWANG  (Griffin  et  ah,  2014)) 
to  create  the  synthetic  cohort  groups  used  in  Part  II  of  the  retention  analysis.  Table  G.  1  gives  an 
overview  of  the  synthetic  cohort  groups  created. 


Table  G.l.  Synthetic  Cohort  Groups  Created  for  Retention  Analysis  Part  II 


Cohort  Group... 

Weighted  to  Look  Like... 

Creates  the... 

Policy  Cohort  Group 

Early  Pre-Policy  Cohort  Group 

Synthetic  Early  Pre-Policy  Cohort  Group 

Policy  Cohort  Group 

Later  Pre-Policy  Cohort  Group 

Synthetic  Later  Pre-Policy  Cohort  Group 

Policy  Cohort  Group 

Combination  Pre-Policy  Cohort 
Group 

Synthetic  Combination  Pre-Policy  Cohort 
Group 

Early  Pre-Policy  Cohort  Group 

Policy  Cohort  Group 

Synthetic  Policy  Cohort  Group  1 

Later  Pre-Policy  Cohort  Group 

Policy  Cohort  Group 

Synthetic  Policy  Cohort  Group  2 

Combination  Pre-Policy 

Policy  Cohort  Group 

Synthetic  Policy  Cohort  Group  3 

Cohort  Group 

The  prior-career-pro  (lie  descriptors  and  all  characteristics  listed  in  Table  6.2,  with  the 
exception  of  Cohort  Year  and  LN(Compensation),  are  used  to  create  the  synthetic  cohort  groups. 
Cohort  Year  is  not  included  because  it  is  impossible  to  weight  one  cohort  group  to  match  the 
cohort  year  distribution  of  another  cohort  group.  LN(Compensation)  is  not  included  because,  in 
general,  nominal  compensation  levels  tend  to  increase  over  time,  so  it  is  inaccurate  to  weight  one 
cohort  group  to  match  the  compensation  distribution  of  another  cohort  group.  The  Pay  Plan 
categories  are  adjusted  for  the  creation  of  the  synthetic  cohorts,  the  “Broad  GS”  Pay  Plan  group 
and  the  “NSPS”  Pay  Plan  group  are  combined  into  one,  which  is  called  the  “Broad  GS  or  NSPS” 
Pay  Plan  group. 

To  illustrate  how  the  weighting  process  works,  the  following  explanation  uses  the  creation  of 
the  synthetic  later  pre-policy  cohort  group  as  an  example.  The  same  general  process  holds  for  the 
creation  of  all  synthetic  cohort  groups. 

To  create  the  synthetic  later  pre-policy  cohort  group,  RAND’s  TWANG  compares  the  policy 
cohort  group  with  the  later  pre-policy  cohort  group  and  assigns  each  observation  (i.e.,  each 
civilian  DAW  entrant)  in  these  groups  with  a  propensity  score.100  In  this  case,  the  propensity 
score,  p(x),  is  the  probability  that  that  individual  is  in  the  later  pre-policy  cohort,  given  his  or  her 

100  The  descriptions  of  propensity-score-based-weighting  presented  in  this  study  are  based  on  the  explanations  from 
Griffin  and  McCaffrey  (2015b),  Griffin  and  McCaffrey  (2015a),  and  McCaffrey,  Ridgeway  and  Morral  (2004). 
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observable  characteristics  (as  seen  in  Equation  B.4  in  Appendix  B)  (Griffin  and  McCaffrey, 
2015b;  Griffin  and  McCaffrey,  2015a;  McCaffrey,  Ridgeway  and  Morral,  2004).  In  general,  the 
observations  in  the  later  pre-policy  cohort  group  should  receive  higher  propensity  scores  (Griffin 
and  McCaffrey,  2015b;  Griffin  and  McCaffrey,  2015a;  McCaffrey,  Ridgeway  and  Morral,  2004). 
Then,  RAND’s  TWANG  applies  weights  to  the  policy  cohort  group  only,  based  on  these 
propensity  scores,  such  that  the  propensity  score  distribution  for  the  observations  in  the  policy 
cohort  group  looks  more  like  that  of  the  later  pre-policy  cohort  group.  Specifically,  each 
observation  (i)  in  the  policy  cohort  group  (c)  will  receive  a  weight  that  equals  that  observation’s 
propensity  score  divided  by  one  minus  that  observation’s  propensity  score  (see  Equation  G.l). 
Observations  in  the  later  pre-policy  cohort  group  do  not  receive  a  weight  (McCaffrey,  Ridgeway 
and  Morral,  2004;  Griffin  and  McCaffrey,  2015b;  Griffin  and  McCaffrey,  2015a). 

Equation  G.  1.  Applied  weights  for  creating  synthetic  cohorts 101 

PiW 

W;c  (1  -PiM) 

Observations  in  the  policy  cohort  group  with  higher  propensity  scores  will  receive  higher 
weights,  while  those  with  lower  propensity  scores  will  receive  lower  weights  (McCaffrey, 
Ridgeway  and  Morral,  2004;  Griffin  and  McCaffrey,  2015b;  Griffin  and  McCaffrey,  2015a). 

This  policy  cohort  group  with  weights  (weighted  to  resemble  the  later  pre-policy  cohort  group)  is 
called  the  synthetic  later  pre-policy  cohort  group. 

Sections  G.1-G.6,  below,  show  the  results  from  the  weighting  process  for  the  creation  of  all 
six  synthetic  cohorts.  In  general,  the  results  show  that  applied  propensity-score  weights  are 
effective.  None  of  the  variables  used  has  an  “Absolute  standard  difference”  102  in  means  between 
the  synthetic  cohort  group  and  the  actual  cohort  group  it  is  weighted  to  resemble  greater  than  0.2, 
which  is  the  threshold  referenced  in  the  RAND  TWANG  (Griffin  et  ah,  2014;  Griffin  and 
McCaffrey,  2015d;  Griffin  and  McCaffrey,  2015c).103  In  fact,  after  weighting,  there  are  no  cases 
where  the  absolute  standard  difference  in  means  is  greater  than  0.1. 

The  t-tests  show  that  some  of  the  differences  in  the  variable  means  between  the  synthetic 
cohort  group  and  the  actual  cohort  group  are  no  longer  statistically  significant  after  weighting. 
Some  differences  remain  statistically  significant.  This  statistical  significance  is  likely  due  to  the 


101  This  equation  stems  from  McCaffrey,  Ridgeway  and  Morral  (2004)  (pp.  405-406),  Griffin  and  McCaffrey 
(2015b),  and  Griffin  and  McCaffrey  (2015a)  (p.  44).  This  is  the  weighting  scheme  used  when  implementing 
weights  to  estimate  the  average  treatment  effect  on  the  treated  (ATT)  (McCaffrey,  Ridgeway  and  Morral,  2004,  pp. 
405-406;  Griffin  and  McCaffrey,  2015b;  Griffin  and  McCaffrey,  2015a,  p.  44).  In  this  case,  the  later  pre-policy 
cohort  group  is  the  quasi-treatment  group,  so  the  policy  cohort  group  is  weighted  to  look  like  the  later  pre-policy 
cohort  group. 

102 

This  phrase  stems  from  Griffin  and  McCaffrey  (2015c)  (p.  28).  The  phrase  is  used  throughout  the  following 
section. 

103 

Some  researchers  use  a  stricter  threshold  of  0.1  (Austin  and  Stuart,  2015). 
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large  number  of  observations  included;  with  so  many  observations,  a  small  difference  in  means 
is  likely  to  be  statistically  significant. 

The  tables  and  figures  presented  in  sections  G.1-C.6  come  from  the  output  of  RAND’s 
TWANG  (Griffin  et  al.,  2014). 
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G.1  Creating  the  Synthetic  Early  Pre-Policy  Cohort  Group  from  the  Policy 
Cohort  Group 


Table  G.1.1.  Means  and  Standard  Deviations  Before  and  After  Propensity-Score  Weighting 


Unweighted  Weighted 

Weighted  Policy 
Cohort  Group 

Early  Pre-Policy  Cohort  (Synthetic  Early  Pre- 

Group  Policy  Cohort  Group  Policy  Cohort  Group) 


Balance  Factors 

Mean 

SD 

Mean 

SD 

Abs  Std 
Diff 

Mean 

SD 

Abs 

Std 

Diff 

1 

PRIOR  CAREER  PRFL:  AD 

0.107 

0.309 

0.177 

0.382 

0.226 

0.111 

0.314 

0.013 

2 

PRIOR  CAREER  PRFL:  AD+DoDCiv 

0.085 

0.279 

0.083 

0.276 

0.006 

0.089 

0.285 

0.015 

3 

PRIOR  CAREER  PRFL:  AD+MilDAW 

0.027 

0.161 

0.05 

0.219 

0.148 

0.028 

0.165 

0.009 

4 

PRIOR  CAREER  PRFL:  DoDCiv 

0.396 

0.489 

0.164 

0.371 

0.473 

0.36 

0.48 

0.074 

5 

PRIOR  CAREER  PRFL:  DoDCiv+CivDAW 

0.119 

0.324 

0.074 

0.261 

0.14 

0.126 

0.331 

0.019 

6 

PRIOR  CAREER  PRFL:  No  DoD  Exp 

0.242 

0.428 

0.412 

0.492 

0.398 

0.26 

0.439 

0.042 

7 

PRIOR  CAREER  PRFL:  Other  DoD  Exp 

0.024 

0.153 

0.038 

0.192 

0.093 

0.026 

0.16 

0.014 

8 

AGE 

40.347 

11.006 

39.372 

11.638 

0.089 

40.179 

11.205 

0.015 

9 

RACE:  Other 

0.23 

0.421 

0.21 

0.407 

0.048 

0.225 

0.417 

0.013 

10 

RACE:  White 

0.77 

0.421 

0.79 

0.407 

0.048 

0.775 

0.417 

0.013 

11 

EDUCATION:  BA 

0.454 

0.498 

0.477 

0.499 

0.045 

0.459 

0.498 

0.01 

12 

EDUCATION:  Less  than  BA 

0.331 

0.471 

0.241 

0.427 

0.193 

0.324 

0.468 

0.015 

13 

EDUCATION:  More  than  BA 

0.215 

0.411 

0.283 

0.45 

0.166 

0.217 

0.412 

0.005 

14 

DOD  ORG:  Air  Force 

0.151 

0.358 

0.191 

0.393 

0.111 

0.147 

0.355 

0.011 

15 

DOD  ORG:  Army 

0.413 

0.492 

0.323 

0.468 

0.181 

0.385 

0.487 

0.056 

16 

DOD  ORG:  Navy 

0.342 

0.475 

0.332 

0.471 

0.023 

0.366 

0.482 

0.049 

17 

DOD  ORG:  Other 

0.094 

0.292 

0.154 

0.361 

0.207 

0.102 

0.302 

0.027 

18 

REGION:  East  North  Central 

0.105 

0.306 

0.135 

0.342 

0.099 

0.109 

0.312 

0.015 

19 

REGION:  East  South  Central 

0.084 

0.278 

0.092 

0.289 

0.027 

0.079 

0.269 

0.02 

20 

REGION:  Mid  Atlantic 

0.1 

0.3 

0.086 

0.281 

0.046 

0.094 

0.292 

0.02 

21 

REGION:  Mountain 

0.065 

0.247 

0.053 

0.224 

0.049 

0.064 

0.245 

0.003 

22 

REGION:  New  England 

0.04 

0.196 

0.035 

0.185 

0.025 

0.04 

0.195 

0.002 

23 

REGION:  Other  or  Missing 

0.029 

0.169 

0.023 

0.149 

0.039 

0.026 

0.16 

0.019 

24 

REGION:  Pacific 

0.151 

0.358 

0.127 

0.333 

0.066 

0.153 

0.36 

0.006 

25 

REGION:  South  Atlantic 

0.339 

0.473 

0.375 

0.484 

0.077 

0.361 

0.48 

0.046 

26 

REGION:  West  North  Central 

0.014 

0.116 

0.012 

0.107 

0.017 

0.012 

0.111 

0.01 

27 

REGION:  West  South  Central 

0.073 

0.26 

0.061 

0.24 

0.044 

0.061 

0.24 

0.043 

28 

GENDER:  Female 

0.36 

0.48 

0.325 

0.468 

0.073 

0.351 

0.477 

0.02 

29 

GENDER:  Male 

0.64 

0.48 

0.675 

0.468 

0.073 

0.649 

0.477 

0.02 

30 

POSITION  TYPE:  Administrative 

0.4 

0.49 

0.405 

0.491 

0.012 

0.383 

0.486 

0.034 

31 

POSITION  TYPE:  Other 

0.014 

0.117 

0.009 

0.093 

0.043 

0.018 

0.132 

0.034 

32 

POSITION  TYPE:  Professional 

0.536 

0.499 

0.535 

0.499 

0.001 

0.553 

0.497 

0.034 

33 

POSITION  TYPE:  Technical 

0.051 

0.22 

0.051 

0.219 

0.003 

0.047 

0.211 

0.02 

34 

SUPERVISOR:  All  Other  Positions 

0.919 

0.273 

0.932 

0.252 

0.048 

0.931 

0.253 

0.045 

35 

SUPERVISOR:  Supvisor/Mngr/Ldr 

0.081 

0.273 

0.068 

0.252 

0.048 

0.069 

0.253 

0.045 
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Unweighted 


Weighted 


Weighted  Policy 
Cohort  Group 

Early  Pre-Policy  Cohort  (Synthetic  Early  Pre- 

Group  Policy  Cohort  Group  Policy  Cohort  Group) 


Balance  Factors 

Mean 

SD 

Mean 

SD 

Abs  Std 
Diff 

Mean 

SD 

Abs 

Std 

Diff 

36 

ACQ  CAREER  FLD:  Business  (CE  or  FM) 

0.088 

0.284 

0.075 

0.264 

0.046 

0.081 

0.273 

0.026 

37 

ACQ  CAREER  FLD:  Contracting 

0.151 

0.358 

0.199 

0.399 

0.134 

0.16 

0.366 

0.024 

38 

ACQ  CAREER  FLD:  Engineering 

0.289 

0.453 

0.213 

0.409 

0.168 

0.305 

0.46 

0.036 

39 

ACQ  CAREER  FLD:  Facilities  Engineering 

0.028 

0.165 

0.063 

0.242 

0.209 

0.03 

0.17 

0.01 

40 

ACQ  CAREER  FLD:  Information  Tech 

0.071 

0.257 

0.053 

0.224 

0.07 

0.061 

0.24 

0.038 

41 

ACQ  CAREER  FLD:  Life  Cycle  Logistics 

0.096 

0.295 

0.141 

0.348 

0.152 

0.096 

0.295 

0.002 

42 

ACQ  CAREER  FLD:  Other 

0.038 

0.192 

0.026 

0.159 

0.065 

0.042 

0.2 

0.019 

43 

ACQ  CAREER  FLD:  Prod,  Quality  &  Manuf 

0.044 

0.205 

0.062 

0.242 

0.09 

0.043 

0.202 

0.006 

44 

ACQ  CAREER  FLD:  Program  Management 

0.087 

0.281 

0.092 

0.29 

0.021 

0.078 

0.269 

0.03 

45 

ACQ  CAREER  FLD:  Test  and  Eval 

0.051 

0.219 

0.039 

0.195 

0.051 

0.052 

0.223 

0.008 

46 

ACQ  CAREER  FLD:  Unknown/Missing 

0.057 

0.232 

0.036 

0.186 

0.092 

0.052 

0.221 

0.024 

47 

RETIREMENT  PLAN:  CSRS 

0.201 

0.401 

0.035 

0.184 

0.414 

0.163 

0.37 

0.094 

48 

RETIREMENT  PLAN:  FERS 

0.793 

0.405 

0.955 

0.207 

0.4 

0.832 

0.374 

0.096 

49 

RETIREMENT  PLAN:  Other 

0.006 

0.075 

0.01 

0.099 

0.055 

0.005 

0.069 

0.012 

50 

CAREER  LEVEL:  Entry-Level 

0.265 

0.441 

0.295 

0.456 

0.07 

0.283 

0.451 

0.042 

51 

CAREER  LEVEL:  Mid-Level 

0.663 

0.473 

0.597 

0.49 

0.138 

0.65 

0.477 

0.026 

52 

CAREER  LEVEL:  Sen/Exec 

0.064 

0.244 

0.099 

0.299 

0.145 

0.059 

0.236 

0.018 

53 

CAREER  LEVEL:  Uncategorized 

0.009 

0.095 

0.008 

0.09 

0.01 

0.007 

0.085 

0.02 

54 

YRS  TO  RET  ELIG:  <5  Yrs  to  Go 

0.082 

0.274 

0.068 

0.252 

0.049 

0.08 

0.271 

0.008 

55 

YRS  TO  RET  ELIG:  10  to  19  Yrs  to  Go 

0.337 

0.473 

0.302 

0.459 

0.073 

0.325 

0.468 

0.025 

56 

YRS  TO  RET  ELIG:  20+  YRs  to  Go 

0.392 

0.488 

0.487 

0.5 

0.195 

0.412 

0.492 

0.041 

57 

YRS  TO  RET  ELIG:  5-9  Yrs  to  Go 

0.145 

0.352 

0.111 

0.315 

0.094 

0.139 

0.346 

0.016 

58 

YRS  TO  RET  ELIG:  Ret  Eligible 

0.045 

0.207 

0.031 

0.173 

0.068 

0.045 

0.206 

0.002 

59 

PAY  PLAN:  Broad  GS  or  NSPS 

0.802 

0.398 

0.847 

0.36 

0.113 

0.794 

0.405 

0.022 

60 

PAY  PLAN:  Demo 

0.19 

0.392 

0.139 

0.346 

0.13 

0.2 

0.4 

0.027 

61 

PAY  PLAN:  Other 

0.008 

0.09 

0.014 

0.117 

0.064 

0.006 

0.078 

0.023 

62 

FICAP:  Handicapped 

0.087 

0.282 

0.082 

0.275 

0.018 

0.086 

0.281 

0.003 

63 

HCAP:  Not  Handicapped 

0.913 

0.282 

0.918 

0.275 

0.018 

0.914 

0.281 

0.003 
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Figure  G.1.1.  All  Absolute  Standard  Differences  in  Means  are  Below  0.1  After  Weighting 


Figure  G.1.2.  After  Weighting,  Some  Differences  in  Means  are  No  Longer  Statistically  Significant 


0.8 


Rank  of  p-value  rank  for  pretreatment  variables 
(hollow  is  weighted,  solid  is  unweighted) 
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G.2  Creating  the  Synthetic  Later  Pre-Policy  Cohort  Group  from  the  Policy 
Cohort  Group 


Table  G.2.1.  Means  and  Standard  Deviations  Before  and  After  Propensity-Score  Weighting 


Unweighted  Weighted 

Weighted  Policy 
Cohort  Group 

Later  Pre-Policy  Cohort  (Synthetic  Later  Pre- 


Group 

Policy  Cohort  Group 

Policy  Cohort  Group) 

Balance  Factors 

Mean 

SD 

Mean 

SD 

Abs  Std 

Diff 

Mean 

SD 

Abs  Std 
Diff 

1 

PRIOR  CAREER  PRFL:  AD 

0.124 

0.329 

0.177 

0.382 

0.162 

0.128 

0.334 

0.012 

2 

PRIOR  CAREER  PRFL:  AD+DoDCiv 

0.1 

0.299 

0.083 

0.276 

0.054 

0.104 

0.305 

0.015 

3 

PRIOR  CAREER  PRFL:  AD+MilDAW 

0.042 

0.2 

0.05 

0.219 

0.044 

0.041 

0.199 

0.001 

4 

PRIOR  CAREER  PRFL:  DoDCiv 

0.312 

0.463 

0.164 

0.371 

0.318 

0.295 

0.456 

0.036 

5 

PRIOR  CAREER  PRFL:  DoDCiv+CivDAW 

0.117 

0.322 

0.074 

0.261 

0.136 

0.117 

0.321 

0.002 

6 

PRIOR  CAREER  PRFL:  No  DoD  Exp 

0.273 

0.446 

0.412 

0.492 

0.312 

0.281 

0.449 

0.017 

7 

PRIOR  CAREER  PRFL:  Other  DoD  Exp 

0.033 

0.177 

0.038 

0.192 

0.033 

0.034 

0.182 

0.009 

8 

AGE 

39.257 

1 1 .492 

39.372 

11.638 

0.01 

39.228 

11.536 

0.002 

9 

RACE:  Other 

0.223 

0.416 

0.21 

0.407 

0.031 

0.221 

0.415 

0.005 

10 

RACE:  White 

0.777 

0.416 

0.79 

0.407 

0.031 

0.779 

0.415 

0.005 

11 

EDUCATION:  BA 

0.471 

0.499 

0.477 

0.499 

0.011 

0.473 

0.499 

0.005 

12 

EDUCATION:  Less  than  BA 

0.289 

0.453 

0.241 

0.427 

0.107 

0.284 

0.451 

0.011 

13 

EDUCATION:  More  than  BA 

0.24 

0.427 

0.283 

0.45 

0.1 

0.243 

0.429 

0.006 

14 

DOD  ORG:  Air  Force 

0.151 

0.358 

0.191 

0.393 

0.112 

0.152 

0.359 

0.002 

15 

DOD  ORG:  Army 

0.449 

0.497 

0.323 

0.468 

0.252 

0.433 

0.496 

0.031 

16 

DOD  ORG:  Navy 

0.254 

0.435 

0.332 

0.471 

0.179 

0.268 

0.443 

0.032 

17 

DOD  ORG:  Other 

0.147 

0.354 

0.154 

0.361 

0.021 

0.147 

0.355 

0.002 

18 

REGION:  East  North  Central 

0.126 

0.332 

0.135 

0.342 

0.028 

0.125 

0.331 

0.002 

19 

REGION:  East  South  Central 

0.088 

0.283 

0.092 

0.289 

0.014 

0.089 

0.285 

0.004 

20 

REGION:  Mid  Atlantic 

0.117 

0.321 

0.086 

0.281 

0.094 

0.117 

0.321 

0 

21 

REGION:  Mountain 

0.066 

0.249 

0.053 

0.224 

0.054 

0.065 

0.246 

0.005 

22 

REGION:  New  England 

0.034 

0.182 

0.035 

0.185 

0.006 

0.035 

0.183 

0.003 

23 

REGION:  Other  or  Missing 

0.025 

0.155 

0.023 

0.149 

0.013 

0.025 

0.156 

0.001 

24 

REGION:  Pacific 

0.116 

0.32 

0.127 

0.333 

0.037 

0.116 

0.321 

0.003 

25 

REGION:  South  Atlantic 

0.349 

0.477 

0.375 

0.484 

0.056 

0.352 

0.478 

0.006 

26 

REGION:  West  North  Central 

0.016 

0.126 

0.012 

0.107 

0.036 

0.014 

0.117 

0.018 

27 

REGION:  West  South  Central 

0.064 

0.245 

0.061 

0.24 

0.011 

0.063 

0.242 

0.006 

28 

GENDER:  Female 

0.354 

0.478 

0.325 

0.468 

0.06 

0.355 

0.478 

0.002 

29 

GENDER:  Male 

0.646 

0.478 

0.675 

0.468 

0.06 

0.645 

0.478 

0.002 

30 

POSITION  TYPE:  Administrative 

0.39 

0.488 

0.405 

0.491 

0.031 

0.396 

0.489 

0.012 

31 

POSITION  TYPE:  Other 

0.011 

0.106 

0.009 

0.093 

0.024 

0.011 

0.104 

0.005 

32 

POSITION  TYPE:  Professional 

0.543 

0.498 

0.535 

0.499 

0.015 

0.538 

0.499 

0.009 

33 

POSITION  TYPE:  Technical 

0.056 

0.23 

0.051 

0.219 

0.023 

0.055 

0.228 

0.004 

34 

SUPERVISOR:  All  Other  Positions 

0.918 

0.274 

0.932 

0.252 

0.05 

0.922 

0.269 

0.012 

35 

SUPERVISOR:  Supvisor/Mngr/Ldr 

0.082 

0.274 

0.068 

0.252 

0.05 

0.078 

0.269 

0.012 

36 

ACQ  CAREER  FLD:  Business  (CE  or  FM) 

0.075 

0.263 

0.075 

0.264 

0.003 

0.077 

0.266 

0.008 
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Unweighted  Weighted 

Weighted  Policy 
Cohort  Group 

Later  Pre-Policy  Cohort  (Synthetic  Later  Pre- 


Group 

Policy  Cohort  Group 

Policy  Cohort  Group) 

Balance  Factors 

Mean 

SD 

Mean 

SD 

Abs  Std 

Diff 

Mean 

SD 

Abs  Std 
Diff 

37 

ACQ  CAREER  FLD:  Contracting 

0.168 

0.374 

0.199 

0.399 

0.083 

0.176 

0.381 

0.022 

38 

ACQ  CAREER  FLD:  Engineering 

0.216 

0.412 

0.213 

0.409 

0.008 

0.221 

0.415 

0.012 

39 

ACQ  CAREER  FLD:  Facilities  Engineering 

0.074 

0.262 

0.063 

0.242 

0.044 

0.062 

0.24 

0.048 

40 

ACQ  CAREER  FLD:  Information  Tech 

0.04 

0.195 

0.053 

0.224 

0.069 

0.041 

0.197 

0.005 

41 

ACQ  CAREER  FLD:  Life  Cycle  Logistics 

0.127 

0.333 

0.141 

0.348 

0.041 

0.129 

0.335 

0.005 

42 

ACQ  CAREER  FLD:  Other 

0.03 

0.171 

0.026 

0.159 

0.026 

0.033 

0.18 

0.018 

43 

ACQ  CAREER  FLD:  Prod,  Quality  &  Manuf 

0.073 

0.26 

0.062 

0.242 

0.04 

0.076 

0.265 

0.011 

44 

ACQ  CAREER  FLD:  Program  Management 

0.074 

0.261 

0.092 

0.29 

0.071 

0.074 

0.262 

0.001 

45 

ACQ  CAREER  FLD:  Test  and  Eval 

0.043 

0.203 

0.039 

0.195 

0.017 

0.044 

0.205 

0.004 

46 

ACQ  CAREER  FLD:  Unknown/Missing 

0.08 

0.272 

0.036 

0.186 

0.164 

0.068 

0.252 

0.046 

47 

RETIREMENT  PLAN:  CSRS 

0.109 

0.312 

0.035 

0.184 

0.239 

0.094 

0.292 

0.048 

48 

RETIREMENT  PLAN:  FERS 

0.882 

0.322 

0.955 

0.207 

0.226 

0.897 

0.304 

0.046 

49 

RETIREMENT  PLAN:  Other 

0.008 

0.091 

0.01 

0.099 

0.017 

0.008 

0.091 

0.001 

50 

CAREER  LEVEL:  Entry-Level 

0.312 

0.463 

0.295 

0.456 

0.035 

0.321 

0.467 

0.021 

51 

CAREER  LEVEL:  Mid-Level 

0.604 

0.489 

0.597 

0.49 

0.014 

0.594 

0.491 

0.02 

52 

CAREER  LEVEL:  Sen/Exec 

0.074 

0.261 

0.099 

0.299 

0.097 

0.077 

0.266 

0.011 

53 

CAREER  LEVEL:  Uncategorized 

0.01 

0.101 

0.008 

0.09 

0.022 

0.008 

0.088 

0.026 

54 

YRS  TO  RET  ELIG:  <5  Yrs  to  Go 

0.075 

0.263 

0.068 

0.252 

0.024 

0.073 

0.26 

0.006 

55 

YRS  TO  RET  ELIG:  10  to  19  Yrs  to  Go 

0.312 

0.463 

0.302 

0.459 

0.022 

0.313 

0.464 

0.001 

56 

YRS  TO  RET  ELIG:  20+  YRs  to  Go 

0.455 

0.498 

0.487 

0.5 

0.065 

0.46 

0.498 

0.009 

57 

YRS  TO  RET  ELIG:  5-9  Yrs  to  Go 

0.118 

0.323 

0.111 

0.315 

0.021 

0.115 

0.319 

0.011 

58 

YRS  TO  RET  ELIG:  Ret  Eligible 

0.04 

0.195 

0.031 

0.173 

0.045 

0.04 

0.196 

0.001 

59 

PAY  PLAN:  Broad  GS  or  NSPS 

0.821 

0.383 

0.847 

0.36 

0.068 

0.826 

0.379 

0.013 

60 

PAY  PLAN:  Demo 

0.169 

0.375 

0.139 

0.346 

0.08 

0.166 

0.372 

0.007 

61 

PAY  PLAN:  Other 

0.01 

0.098 

0.014 

0.117 

0.042 

0.008 

0.087 

0.022 

62 

HCAP:  Handicapped 

0.078 

0.269 

0.082 

0.275 

0.014 

0.079 

0.27 

0.003 

63 

HCAP:  Not  Handicapped 

0.922 

0.269 

0.918 

0.275 

0.014 

0.921 

0.27 

0.003 
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Figure  G.2.1.  All  Absolute  Standard  Differences  in  Means  are  Below  0.1  After  Weighting 


Figure  G.2.2.  After  Weighting,  Some  Differences  in  Means  are  No  Longer  Statistically  Significant 
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G.3  Creating  the  Synthetic  Combination  Pre-Policy  Cohort  Group  from  the 
Policy  Cohort  Group 


Table  G.3.1.  Means  and  Standard  Deviations  Before  and  After  Propensity-Score  Weighting 


Unweighted 

Weighted 

Combination  Pre-Policy 
Cohort  Group 

Policy  Cohort  Group 

Weighted  Policy  Cohort 
Group  (Synthetic 
Combination  Pre-Policy 
Cohort  Group) 

Balance  Factors 

Mean 

SD 

Mean 

SD 

Abs  Std 

Diff 

Mean 

SD 

Abs  Std 
Diff 

PRIOR  CAREER  PRFL:  AD 

0.116 

0.32 

0.177 

0.382 

0.192 

0.121 

0.326 

0.015 

PRIOR  CAREER  PRFL:  AD+DoDCiv 

0.092 

0.29 

0.083 

0.276 

0.031 

0.095 

0.293 

0.008 

PRIOR  CAREER  PRFL:  AD+MilDAW 

0.034 

0.182 

0.05 

0.219 

0.089 

0.036 

0.187 

0.01 

PRIOR  CAREER  PRFL:  DoDCiv 

0.353 

0.478 

0.164 

0.371 

0.394 

0.33 

0.47 

0.047 

PRIOR  CAREER  PRFL:  DoDCiv+CivDAW 

0.118 

0.323 

0.074 

0.261 

0.138 

0.118 

0.323 

0.001 

PRIOR  CAREER  PRFL:  No  DoD  Exp 

0.258 

0.438 

0.412 

0.492 

0.353 

0.27 

0.444 

0.026 

PRIOR  CAREER  PRFL:  Other  DoD  Exp 

0.028 

0.166 

0.038 

0.192 

0.06 

0.031 

0.173 

0.014 

AGE 

39.788 

11.271 

39.372 

11.638 

0.037 

39.664 

1 1 .384 

0.011 

RACE:  Other 

0.226 

0.418 

0.21 

0.407 

0.039 

0.223 

0.416 

0.008 

RACE:  White 

0.774 

0.418 

0.79 

0.407 

0.039 

0.777 

0.416 

0.008 

EDUCATION:  BA 

0.463 

0.499 

0.477 

0.499 

0.028 

0.468 

0.499 

0.011 

EDUCATION:  Less  than  BA 

0.31 

0.462 

0.241 

0.427 

0.149 

0.305 

0.46 

0.011 

EDUCATION:  More  than  BA 

0.228 

0.419 

0.283 

0.45 

0.132 

0.227 

0.419 

0.001 

DOD  ORG:  Air  Force 

0.151 

0.358 

0.191 

0.393 

0.111 

0.147 

0.354 

0.011 

DOD  ORG:  Army 

0.431 

0.495 

0.323 

0.468 

0.218 

0.413 

0.492 

0.037 

DOD  ORG:  Navy 

0.297 

0.457 

0.332 

0.471 

0.076 

0.316 

0.465 

0.043 

DOD  ORG:  Other 

0.121 

0.326 

0.154 

0.361 

0.102 

0.124 

0.329 

0.009 

REGION:  East  North  Central 

0.116 

0.32 

0.135 

0.342 

0.061 

0.12 

0.324 

0.013 

REGION:  East  South  Central 

0.086 

0.28 

0.092 

0.289 

0.02 

0.085 

0.278 

0.005 

REGION:  Mid  Atlantic 

0.109 

0.311 

0.086 

0.281 

0.072 

0.106 

0.308 

0.008 

REGION:  Mountain 

0.066 

0.248 

0.053 

0.224 

0.052 

0.064 

0.245 

0.005 

REGION:  New  England 

0.037 

0.189 

0.035 

0.185 

0.009 

0.037 

0.189 

0.001 

REGION:  Other  or  Missing 

0.027 

0.162 

0.023 

0.149 

0.026 

0.025 

0.157 

0.011 

REGION:  Pacific 

0.133 

0.339 

0.127 

0.333 

0.016 

0.133 

0.34 

0.001 

REGION:  South  Atlantic 

0.344 

0.475 

0.375 

0.484 

0.066 

0.357 

0.479 

0.027 

REGION:  West  North  Central 

0.015 

0.121 

0.012 

0.107 

0.027 

0.013 

0.113 

0.017 

REGION:  West  South  Central 

0.068 

0.252 

0.061 

0.24 

0.027 

0.06 

0.238 

0.031 

GENDER:  Female 

0.357 

0.479 

0.325 

0.468 

0.066 

0.354 

0.478 

0.007 

GENDER:  Male 

0.643 

0.479 

0.675 

0.468 

0.066 

0.646 

0.478 

0.007 

POSITION  TYPE:  Administrative 

0.395 

0.489 

0.405 

0.491 

0.022 

0.388 

0.487 

0.013 

POSITION  TYPE:  Other 

0.013 

0.111 

0.009 

0.093 

0.034 

0.012 

0.107 

0.008 

POSITION  TYPE:  Professional 

0.539 

0.498 

0.535 

0.499 

0.008 

0.547 

0.498 

0.016 

POSITION  TYPE:  Technical 

0.054 

0.225 

0.051 

0.219 

0.013 

0.053 

0.224 

0.003 

SUPERVISOR:  All  Other  Positions 

0.919 

0.273 

0.932 

0.252 

0.049 

0.924 

0.264 

0.021 

SUPERVISOR:  Supvisor/Mngr/Ldr 

0.081 

0.273 

0.068 

0.252 

0.049 

0.076 

0.264 

0.021 

ACQ  CAREER  FLD:  Business  (CE  or  FM) 

0.081 

0.273 

0.075 

0.264 

0.022 

0.08 

0.271 

0.006 
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Unweighted  Weighted 

Weighted  Policy  Cohort 
Group  (Synthetic 

Combination  Pre-Policy  Combination  Pre-Policy 

Cohort  Group  Policy  Cohort  Group  Cohort  Group) 

Abs  Std  Abs  Std 


Balance  Factors 

Mean 

SD 

Mean 

SD 

Diff 

Mean 

SD 

Diff 

37 

ACQ  CAREER  FLD:  Contracting 

0.16 

0.366 

0.199 

0.399 

0.108 

0.168 

0.374 

0.024 

38 

ACQ  CAREER  FLD:  Engineering 

0.251 

0.434 

0.213 

0.409 

0.089 

0.261 

0.439 

0.023 

39 

ACQ  CAREER  FLD:  Facilities  Engineering 

0.052 

0.221 

0.063 

0.242 

0.05 

0.047 

0.211 

0.022 

40 

ACQ  CAREER  FLD:  Information  Tech 

0.055 

0.228 

0.053 

0.224 

0.008 

0.051 

0.22 

0.019 

41 

ACQ  CAREER  FLD:  Life  Cycle  Logistics 

0.112 

0.315 

0.141 

0.348 

0.092 

0.114 

0.318 

0.007 

42 

ACQ  CAREER  FLD:  Other 

0.034 

0.182 

0.026 

0.159 

0.046 

0.035 

0.183 

0.002 

43 

ACQ  CAREER  FLD:  Prod,  Quality  &  Manuf 

0.059 

0.235 

0.062 

0.242 

0.015 

0.061 

0.239 

0.009 

44 

ACQ  CAREER  FLD:  Program  Management 

0.08 

0.271 

0.092 

0.29 

0.046 

0.076 

0.266 

0.013 

45 

ACQ  CAREER  FLD:  Test  and  Eval 

0.047 

0.211 

0.039 

0.195 

0.034 

0.048 

0.214 

0.007 

46 

ACQ  CAREER  FLD:  Unknown/Missing 

0.069 

0.254 

0.036 

0.186 

0.131 

0.059 

0.235 

0.041 

47 

RETIREMENT  PLAN:  CSRS 

0.154 

0.361 

0.035 

0.184 

0.33 

0.129 

0.335 

0.07 

48 

RETIREMENT  PLAN:  FERS 

0.839 

0.368 

0.955 

0.207 

0.316 

0.865 

0.342 

0.071 

49 

RETIREMENT  PLAN:  Other 

0.007 

0.083 

0.01 

0.099 

0.034 

0.006 

0.079 

0.009 

50 

CAREER  LEVEL:  Entry-Level 

0.289 

0.453 

0.295 

0.456 

0.014 

0.302 

0.459 

0.029 

51 

CAREER  LEVEL:  Mid-Level 

0.633 

0.482 

0.597 

0.49 

0.073 

0.623 

0.485 

0.019 

52 

CAREER  LEVEL:  Sen/Exec 

0.069 

0.253 

0.099 

0.299 

0.12 

0.068 

0.252 

0.003 

53 

CAREER  LEVEL:  Uncategorized 

0.01 

0.098 

0.008 

0.09 

0.017 

0.007 

0.082 

0.03 

54 

YRS  TO  RET  ELIG:  <5  Yrs  to  Go 

0.078 

0.268 

0.068 

0.252 

0.037 

0.076 

0.265 

0.007 

55 

YRS  TO  RET  ELIG:  10  to  19  Yrs  to  Go 

0.324 

0.468 

0.302 

0.459 

0.047 

0.317 

0.465 

0.015 

56 

YRS  TO  RET  ELIG:  20+  YRs  to  Go 

0.424 

0.494 

0.487 

0.5 

0.127 

0.437 

0.496 

0.025 

57 

YRS  TO  RET  ELIG:  5-9  Yrs  to  Go 

0.131 

0.338 

0.111 

0.315 

0.058 

0.128 

0.334 

0.009 

58 

YRS  TO  RET  ELIG:  Ret  Eligible 

0.042 

0.201 

0.031 

0.173 

0.057 

0.042 

0.199 

0.004 

59 

PAY  PLAN:  Broad  GS  or  NSPS 

0.812 

0.391 

0.847 

0.36 

0.09 

0.808 

0.394 

0.009 

60 

PAY  PLAN:  Demo 

0.179 

0.383 

0.139 

0.346 

0.105 

0.186 

0.389 

0.017 

61 

PAY  PLAN:  Other 

0.009 

0.094 

0.014 

0.117 

0.052 

0.006 

0.078 

0.031 

62 

HCAP:  Handicapped 

0.083 

0.275 

0.082 

0.275 

0.002 

0.082 

0.275 

0.001 

63 

HCAP:  Not  Handicapped 

0.917 

0.275 

0.918 

0.275 

0.002 

0.918 

0.275 

0.001 

135 


Figure  G.3.1.  All  Absolute  Standard  Differences  in  Means  are  Below  0.1  After  Weighting 


Figure  G.3.2.  After  Weighting,  Some  Differences  in  Means  are  No  Longer  Statistically  Significant 
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G.4  Creating  the  Synthetic  Policy  Cohort  Group  from  the  Early  Pre-Policy 
Cohort  Group 


Table  G.4.1.  Means  and  Standard  Deviations  Before  and  After  Propensity-Score  Weighting 


Balance  Factors 

Policy  Cohort  Group 

Mean  SD 

Unweighted 

Early  Pre-Policy  Cohort 
Group 

Mean  SD 

Abs  Std 

Diff 

Weighted 

Weighted  Pre-Policy 
Cohort  Group  (Synthetic 
Policy  Cohort  Group  1) 

Mean  SD 

Abs  Std 
Diff 

PRIOR  CAREER  PRFL:  AD 

0.177 

0.382 

0.107 

0.309 

0.183 

0.175 

0.38 

0.005 

PRIOR  CAREER  PRFL:  AD+DoDCiv 

0.083 

0.276 

0.085 

0.279 

0.006 

0.09 

0.286 

0.023 

PRIOR  CAREER  PRFL:  AD+MilDAW 

0.05 

0.219 

0.027 

0.161 

0.109 

0.055 

0.229 

0.022 

PRIOR  CAREER  PRFL:  DoDCiv 

0.164 

0.371 

0.396 

0.489 

0.625 

0.17 

0.376 

0.015 

PRIOR  CAREER  PRFL:  DoDCiv+CivDAW 

0.074 

0.261 

0.119 

0.324 

0.174 

0.074 

0.262 

0.002 

PRIOR  CAREER  PRFL:  No  DoD  Exp 

0.412 

0.492 

0.242 

0.428 

0.347 

0.394 

0.489 

0.038 

PRIOR  CAREER  PRFL:  Other  DoD  Exp 

0.038 

0.192 

0.024 

0.153 

0.074 

0.042 

0.2 

0.017 

AGE 

39.372 

11.638 

40.347 

1 1 .006 

0.084 

39.308 

11.513 

0.005 

RACE:  Other 

0.21 

0.407 

0.23 

0.421 

0.049 

0.211 

0.408 

0.003 

RACE:  White 

0.79 

0.407 

0.77 

0.421 

0.049 

0.789 

0.408 

0.003 

EDUCATION:  BA 

0.477 

0.499 

0.454 

0.498 

0.045 

0.483 

0.5 

0.014 

EDUCATION:  Less  than  BA 

0.241 

0.427 

0.331 

0.471 

0.212 

0.236 

0.425 

0.011 

EDUCATION:  More  than  BA 

0.283 

0.45 

0.215 

0.411 

0.152 

0.281 

0.449 

0.005 

DOD  ORG:  Air  Force 

0.191 

0.393 

0.151 

0.358 

0.101 

0.186 

0.389 

0.013 

DOD  ORG:  Army 

0.323 

0.468 

0.413 

0.492 

0.191 

0.321 

0.467 

0.004 

DOD  ORG:  Navy 

0.332 

0.471 

0.342 

0.475 

0.023 

0.358 

0.479 

0.056 

DOD  ORG:  Other 

0.154 

0.361 

0.094 

0.292 

0.167 

0.135 

0.342 

0.054 

REGION:  East  North  Central 

0.135 

0.342 

0.105 

0.306 

0.089 

0.13 

0.336 

0.016 

REGION:  East  South  Central 

0.092 

0.289 

0.084 

0.278 

0.026 

0.076 

0.266 

0.053 

REGION:  Mid  Atlantic 

0.086 

0.281 

0.1 

0.3 

0.049 

0.087 

0.282 

0.003 

REGION:  Mountain 

0.053 

0.224 

0.065 

0.247 

0.054 

0.055 

0.228 

0.009 

REGION:  New  England 

0.035 

0.185 

0.04 

0.196 

0.026 

0.037 

0.188 

0.007 

REGION:  Other  or  Missing 

0.023 

0.149 

0.029 

0.169 

0.044 

0.022 

0.145 

0.008 

REGION:  Pacific 

0.127 

0.333 

0.151 

0.358 

0.07 

0.127 

0.333 

0 

REGION:  South  Atlantic 

0.375 

0.484 

0.339 

0.473 

0.075 

0.396 

0.489 

0.043 

REGION:  West  North  Central 

0.012 

0.107 

0.014 

0.116 

0.019 

0.011 

0.102 

0.011 

REGION:  West  South  Central 

0.061 

0.24 

0.073 

0.26 

0.047 

0.059 

0.236 

0.008 

GENDER:  Female 

0.325 

0.468 

0.36 

0.48 

0.075 

0.319 

0.466 

0.014 

GENDER:  Male 

0.675 

0.468 

0.64 

0.48 

0.075 

0.681 

0.466 

0.014 

POSITION  TYPE:  Administrative 

0.405 

0.491 

0.4 

0.49 

0.012 

0.389 

0.488 

0.033 

POSITION  TYPE:  Other 

0.009 

0.093 

0.014 

0.117 

0.054 

0.006 

0.08 

0.025 

POSITION  TYPE:  Professional 

0.535 

0.499 

0.536 

0.499 

0.001 

0.554 

0.497 

0.038 

POSITION  TYPE:  Technical 

0.051 

0.219 

0.051 

0.22 

0.003 

0.05 

0.219 

0.001 

SUPERVISOR:  All  Other  Positions 

0.932 

0.252 

0.919 

0.273 

0.052 

0.933 

0.25 

0.004 

SUPERVISOR:  Supvisor/Mngr/Ldr 

0.068 

0.252 

0.081 

0.273 

0.052 

0.067 

0.25 

0.004 

ACQ  CAREER  FLD:  Business  (CE  or  FM) 

0.075 

0.264 

0.088 

0.284 

0.049 

0.079 

0.27 

0.014 

137 


Unweighted  Weighted 

Weighted  Pre-Policy 

Early  Pre-Policy  Cohort  Cohort  Group  (Synthetic 

Policy  Cohort  Group  Group  Policy  Cohort  Group  1) 

Abs  Std  Abs  Std 


Balance  Factors 

Mean 

SD 

Mean 

SD 

Diff 

Mean 

SD 

Diff 

37 

ACQ  CAREER  FLD:  Contracting 

0.199 

0.399 

0.151 

0.358 

0.12 

0.212 

0.409 

0.033 

38 

ACQ  CAREER  FLD:  Engineering 

0.213 

0.409 

0.289 

0.453 

0.186 

0.225 

0.417 

0.03 

39 

ACQ  CAREER  FLD:  Facilities  Engineering 

0.063 

0.242 

0.028 

0.165 

0.143 

0.068 

0.252 

0.022 

40 

ACQ  CAREER  FLD:  Information  Tech 

0.053 

0.224 

0.071 

0.257 

0.08 

0.047 

0.211 

0.029 

41 

ACQ  CAREER  FLD:  Life  Cycle  Logistics 

0.141 

0.348 

0.096 

0.295 

0.129 

0.135 

0.341 

0.018 

42 

ACQ  CAREER  FLD:  Other 

0.026 

0.159 

0.038 

0.192 

0.079 

0.025 

0.156 

0.005 

43 

ACQ  CAREER  FLD:  Prod,  Quality  &  Manuf 

0.062 

0.242 

0.044 

0.205 

0.076 

0.058 

0.233 

0.019 

44 

ACQ  CAREER  FLD:  Program  Management 

0.092 

0.29 

0.087 

0.281 

0.02 

0.084 

0.278 

0.028 

45 

ACQ  CAREER  FLD:  Test  and  Eval 

0.039 

0.195 

0.051 

0.219 

0.057 

0.04 

0.197 

0.005 

46 

ACQ  CAREER  FLD:  Unknown/Missing 

0.036 

0.186 

0.057 

0.232 

0.114 

0.027 

0.161 

0.049 

47 

RETIREMENT  PLAN:  CSRS 

0.035 

0.184 

0.201 

0.401 

0.904 

0.038 

0.191 

0.015 

48 

RETIREMENT  PLAN:  FERS 

0.955 

0.207 

0.793 

0.405 

0.783 

0.955 

0.207 

0 

49 

RETIREMENT  PLAN:  Other 

0.01 

0.099 

0.006 

0.075 

0.042 

0.007 

0.084 

0.027 

50 

CAREER  LEVEL:  Entry-Level 

0.295 

0.456 

0.265 

0.441 

0.067 

0.31 

0.463 

0.033 

51 

CAREER  LEVEL:  Mid-Level 

0.597 

0.49 

0.663 

0.473 

0.133 

0.593 

0.491 

0.008 

52 

CAREER  LEVEL:  Sen/Exec 

0.099 

0.299 

0.064 

0.244 

0.119 

0.088 

0.283 

0.039 

53 

CAREER  LEVEL:  Uncategorized 

0.008 

0.09 

0.009 

0.095 

0.011 

0.009 

0.093 

0.006 

54 

YRS  TO  RET  ELIG:  <5  Yrs  to  Go 

0.068 

0.252 

0.082 

0.274 

0.054 

0.064 

0.244 

0.018 

55 

YRS  TO  RET  ELIG:  10  to  19  Yrs  to  Go 

0.302 

0.459 

0.337 

0.473 

0.075 

0.31 

0.463 

0.018 

56 

YRS  TO  RET  ELIG:  20+  YRs  to  Go 

0.487 

0.5 

0.392 

0.488 

0.191 

0.484 

0.5 

0.007 

57 

YRS  TO  RET  ELIG:  5-9  Yrs  to  Go 

0.111 

0.315 

0.145 

0.352 

0.105 

0.114 

0.318 

0.008 

58 

YRS  TO  RET  ELIG:  Ret  Eligible 

0.031 

0.173 

0.045 

0.207 

0.082 

0.028 

0.166 

0.015 

59 

PAY  PLAN:  Broad  GS  or  NSPS 

0.847 

0.36 

0.802 

0.398 

0.126 

0.847 

0.36 

0 

60 

PAY  PLAN:  Demo 

0.139 

0.346 

0.19 

0.392 

0.147 

0.143 

0.351 

0.014 

61 

PAY  PLAN:  Other 

0.014 

0.117 

0.008 

0.09 

0.049 

0.009 

0.096 

0.039 

62 

HCAP:  Handicapped 

0.082 

0.275 

0.087 

0.282 

0.019 

0.084 

0.278 

0.007 

63 

HCAP:  Not  Handicapped 

0.918 

0.275 

0.913 

0.282 

0.019 

0.916 

0.278 

0.007 
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Figure  G.4.1.  All  Absolute  Standard  Differences  in  Means  are  Below  0.1  After  Weighting 


Figure  G.4.2.  After  Weighting,  Some  Differences  in  Means  are  No  Longer  Statistically  Significant 
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G.5  Creating  the  Synthetic  Policy  Cohort  Group  from  the  Later  Pre-Policy 
Cohort  Group 


Table  G.5.1.  Means  and  Standard  Deviations  Before  and  After  Propensity-Score  Weighting 


Balance  Factors 

Policy  Cohort  Group 

Mean  SD 

Unweighted 

Later  Pre-Policy  Cohort 
Group 

Mean  SD 

Abs  Std 

Diff 

Weighted 

Weighted  Pre-Policy 
Cohort  Group  (Synthetic 
Policy  Cohort  Group  2) 

Mean  SD 

Abs 

Std  Diff 

PRIOR  CAREER  PRFL:  AD 

0.177 

0.382 

0.124 

0.329 

0.139 

0.183 

0.387 

0.016 

PRIOR  CAREER  PRFL:  AD+DoDCiv 

0.083 

0.276 

0.1 

0.299 

0.059 

0.088 

0.283 

0.017 

PRIOR  CAREER  PRFL:  AD+MilDAW 

0.05 

0.219 

0.042 

0.2 

0.04 

0.055 

0.228 

0.021 

PRIOR  CAREER  PRFL:  DoDCiv 

0.164 

0.371 

0.312 

0.463 

0.397 

0.166 

0.372 

0.005 

PRIOR  CAREER  PRFL:  DoDCiv+CivDAW 

0.074 

0.261 

0.117 

0.322 

0.167 

0.075 

0.264 

0.006 

PRIOR  CAREER  PRFL:  No  DoD  Exp 

0.412 

0.492 

0.273 

0.446 

0.283 

0.39 

0.488 

0.045 

PRIOR  CAREER  PRFL:  Other  DoD  Exp 

0.038 

0.192 

0.033 

0.177 

0.03 

0.042 

0.201 

0.019 

AGE 

39.372 

11.638 

39.257 

1 1 .492 

0.01 

39.176 

11.517 

0.017 

RACE:  Other 

0.21 

0.407 

0.223 

0.416 

0.032 

0.216 

0.412 

0.016 

RACE:  White 

0.79 

0.407 

0.777 

0.416 

0.032 

0.784 

0.412 

0.016 

EDUCATION:  BA 

0.477 

0.499 

0.471 

0.499 

0.011 

0.481 

0.5 

0.01 

EDUCATION:  Less  than  BA 

0.241 

0.427 

0.289 

0.453 

0.113 

0.246 

0.431 

0.012 

EDUCATION:  More  than  BA 

0.283 

0.45 

0.24 

0.427 

0.095 

0.273 

0.445 

0.022 

DOD  ORG:  Air  Force 

0.191 

0.393 

0.151 

0.358 

0.102 

0.177 

0.381 

0.036 

DOD  ORG:  Army 

0.323 

0.468 

0.449 

0.497 

0.268 

0.332 

0.471 

0.018 

DOD  ORG:  Navy 

0.332 

0.471 

0.254 

0.435 

0.166 

0.337 

0.473 

0.012 

DOD  ORG:  Other 

0.154 

0.361 

0.147 

0.354 

0.021 

0.155 

0.362 

0.001 

REGION:  East  North  Central 

0.135 

0.342 

0.126 

0.332 

0.028 

0.128 

0.334 

0.021 

REGION:  East  South  Central 

0.092 

0.289 

0.088 

0.283 

0.013 

0.082 

0.274 

0.035 

REGION:  Mid  Atlantic 

0.086 

0.281 

0.117 

0.321 

0.108 

0.091 

0.287 

0.016 

REGION:  Mountain 

0.053 

0.224 

0.066 

0.249 

0.06 

0.056 

0.231 

0.015 

REGION:  New  England 

0.035 

0.185 

0.034 

0.182 

0.006 

0.036 

0.186 

0.003 

REGION:  Other  or  Missing 

0.023 

0.149 

0.025 

0.155 

0.013 

0.024 

0.152 

0.007 

REGION:  Pacific 

0.127 

0.333 

0.116 

0.32 

0.035 

0.124 

0.329 

0.011 

REGION:  South  Atlantic 

0.375 

0.484 

0.349 

0.477 

0.055 

0.39 

0.488 

0.029 

REGION:  West  North  Central 

0.012 

0.107 

0.016 

0.126 

0.042 

0.011 

0.104 

0.006 

REGION:  West  South  Central 

0.061 

0.24 

0.064 

0.245 

0.011 

0.059 

0.236 

0.008 

GENDER:  Female 

0.325 

0.468 

0.354 

0.478 

0.061 

0.322 

0.467 

0.006 

GENDER:  Male 

0.675 

0.468 

0.646 

0.478 

0.061 

0.678 

0.467 

0.006 

POSITION  TYPE:  Administrative 

0.405 

0.491 

0.39 

0.488 

0.031 

0.396 

0.489 

0.02 

POSITION  TYPE:  Other 

0.009 

0.093 

0.011 

0.106 

0.027 

0.009 

0.096 

0.006 

POSITION  TYPE:  Professional 

0.535 

0.499 

0.543 

0.498 

0.015 

0.543 

0.498 

0.015 

POSITION  TYPE:  Technical 

0.051 

0.219 

0.056 

0.23 

0.024 

0.052 

0.222 

0.007 

SUPERVISOR:  All  Other  Positions 

0.932 

0.252 

0.918 

0.274 

0.054 

0.925 

0.263 

0.028 

SUPERVISOR:  Supvisor/Mngr/Ldr 

0.068 

0.252 

0.082 

0.274 

0.054 

0.075 

0.263 

0.028 

ACQ  CAREER  FLD:  Business  (CE  or  FM) 

0.075 

0.264 

0.075 

0.263 

0.003 

0.077 

0.266 

0.006 
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37 

38 

39 

40 

41 

42 

43 

44 

45 

46 

47 

48 

49 

50 

51 

52 

53 

54 

55 

56 

57 

58 

59 

60 

61 

62 

63 


Unweighted 

Weighted 

Policy  Cohort  Group 

Later  Pre-Policy  Cohort 
Group 

Weighted  Pre-Policy 
Cohort  Group  (Synthetic 
Policy  Cohort  Group  2) 

Balance  Factors 

Mean 

SD 

Mean 

SD 

Abs  Std 

Diff 

Mean 

SD 

Abs 

Std  Diff 

ACQ  CAREER  FLD:  Contracting 

0.199 

0.399 

0.168 

0.374 

0.078 

0.208 

0.406 

0.022 

ACQ  CAREER  FLD:  Engineering 

0.213 

0.409 

0.216 

0.412 

0.008 

0.214 

0.41 

0.003 

ACQ  CAREER  FLD:  Facilities  Engineering 

0.063 

0.242 

0.074 

0.262 

0.047 

0.055 

0.228 

0.032 

ACQ  CAREER  FLD:  Information  Tech 

0.053 

0.224 

0.04 

0.195 

0.06 

0.049 

0.216 

0.019 

ACQ  CAREER  FLD:  Life  Cycle  Logistics 

0.141 

0.348 

0.127 

0.333 

0.039 

0.141 

0.348 

0 

ACQ  CAREER  FLD:  Other 

0.026 

0.159 

0.03 

0.171 

0.028 

0.026 

0.16 

0.003 

ACQ  CAREER  FLD:  Prod,  Quality  &  Manuf 

0.062 

0.242 

0.073 

0.26 

0.043 

0.067 

0.249 

0.017 

ACQ  CAREER  FLD:  Program  Management 

0.092 

0.29 

0.074 

0.261 

0.064 

0.085 

0.279 

0.025 

ACQ  CAREER  FLD:  Test  and  Eval 

0.039 

0.195 

0.043 

0.203 

0.018 

0.04 

0.197 

0.005 

ACQ  CAREER  FLD:  Unknown/Missing 

0.036 

0.186 

0.08 

0.272 

0.239 

0.038 

0.192 

0.013 

RETIREMENT  PLAN:  CSRS 

0.035 

0.184 

0.109 

0.312 

0.406 

0.037 

0.189 

0.012 

RETIREMENT  PLAN:  FERS 

0.955 

0.207 

0.882 

0.322 

0.352 

0.955 

0.208 

0.002 

RETIREMENT  PLAN:  Other 

0.01 

0.099 

0.008 

0.091 

0.016 

0.008 

0.089 

0.018 

CAREER  LEVEL:  Entry-Level 

0.295 

0.456 

0.312 

0.463 

0.036 

0.316 

0.465 

0.045 

CAREER  LEVEL:  Mid-Level 

0.597 

0.49 

0.604 

0.489 

0.014 

0.584 

0.493 

0.026 

CAREER  LEVEL:  Sen/Exec 

0.099 

0.299 

0.074 

0.261 

0.085 

0.093 

0.29 

0.022 

CAREER  LEVEL:  Uncategorized 

0.008 

0.09 

0.01 

0.101 

0.025 

0.007 

0.085 

0.01 

YRS  TO  RET  ELIG:  <5  Yrs  to  Go 

0.068 

0.252 

0.075 

0.263 

0.025 

0.064 

0.246 

0.015 

YRS  TO  RET  ELIG:  10  to  19  Yrs  to  Go 

0.302 

0.459 

0.312 

0.463 

0.022 

0.311 

0.463 

0.019 

YRS  TO  RET  ELIG:  20+  YRs  to  Go 

0.487 

0.5 

0.455 

0.498 

0.064 

0.486 

0.5 

0.002 

YRS  TO  RET  ELIG:  5-9  Yrs  to  Go 

0.111 

0.315 

0.118 

0.323 

0.022 

0.11 

0.313 

0.003 

YRS  TO  RET  ELIG:  Ret  Eligible 

0.031 

0.173 

0.04 

0.195 

0.051 

0.028 

0.165 

0.016 

PAY  PLAN:  Broad  GS  or  NSPS 

0.847 

0.36 

0.821 

0.383 

0.072 

0.848 

0.359 

0.001 

PAY  PLAN:  Demo 

0.139 

0.346 

0.169 

0.375 

0.087 

0.144 

0.351 

0.014 

PAY  PLAN:  Other 

0.014 

0.117 

0.01 

0.098 

0.035 

0.009 

0.092 

0.045 

FICAP:  Flandicapped 

0.082 

0.275 

0.078 

0.269 

0.014 

0.082 

0.274 

0.001 

FICAP:  Not  Flandicapped 

0.918 

0.275 

0.922 

0.269 

0.014 

0.918 

0.274 

0.001 
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Figure  G.5.1.  All  Absolute  Standard  Differences  in  Means  are  Below  0.1  After  Weighting 
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Figure  G.5.2.  After  Weighting,  Most  Differences  in  Means  are  Still  Statistically  Significant 
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G.6  Creating  the  Synthetic  Policy  Cohort  Group  from  the  Combination  Pre- 
Policy  Cohort  Group 


Table  G.6.1.  Means  and  Standard  Deviations  Before  and  After  Propensity-Score  Weighting 


Unweighted 

Weighted 

Policy  Cohort  Group 

Combination  Pre- 
Policy  Cohort  Group 

Weighted  Pre-Policy 
Cohort  Group  (Synthetic 
Policy  Cohort  Group  3) 

Balance  Factors 

Mean 

SD 

Mean 

SD 

Abs  Std 

Diff 

Mean 

SD 

Abs  Std 
Diff 

PRIOR  CAREER  PRFL:  AD 

0.177 

0.382 

0.116 

0.32 

0.161 

0.18 

0.384 

0.006 

PRIOR  CAREER  PRFL:  AD+DoDCiv 

0.083 

0.276 

0.092 

0.29 

0.033 

0.086 

0.281 

0.01 

PRIOR  CAREER  PRFL:  AD+MilDAW 

0.05 

0.219 

0.034 

0.182 

0.074 

0.054 

0.226 

0.017 

PRIOR  CAREER  PRFL:  DoDCiv 

0.164 

0.371 

0.353 

0.478 

0.508 

0.167 

0.373 

0.006 

PRIOR  CAREER  PRFL:  DoDCiv+CivDAW 

0.074 

0.261 

0.118 

0.323 

0.17 

0.077 

0.266 

0.011 

PRIOR  CAREER  PRFL:  No  DoD  Exp 

0.412 

0.492 

0.258 

0.438 

0.314 

0.396 

0.489 

0.034 

PRIOR  CAREER  PRFL:  Other  DoD  Exp 

0.038 

0.192 

0.028 

0.166 

0.052 

0.041 

0.198 

0.012 

AGE 

39.372 

11.638 

39.788 

11.271 

0.036 

39.262 

11.583 

0.009 

RACE:  Other 

0.21 

0.407 

0.226 

0.418 

0.04 

0.212 

0.409 

0.006 

RACE:  White 

0.79 

0.407 

0.774 

0.418 

0.04 

0.788 

0.409 

0.006 

EDUCATION:  BA 

0.477 

0.499 

0.463 

0.499 

0.028 

0.481 

0.5 

0.009 

EDUCATION:  Less  than  BA 

0.241 

0.427 

0.31 

0.462 

0.161 

0.243 

0.429 

0.005 

EDUCATION:  More  than  BA 

0.283 

0.45 

0.228 

0.419 

0.122 

0.276 

0.447 

0.015 

DOD  ORG:  Air  Force 

0.191 

0.393 

0.151 

0.358 

0.101 

0.18 

0.384 

0.027 

DOD  ORG:  Army 

0.323 

0.468 

0.431 

0.495 

0.23 

0.328 

0.469 

0.01 

DOD  ORG:  Navy 

0.332 

0.471 

0.297 

0.457 

0.074 

0.341 

0.474 

0.021 

DOD  ORG:  Other 

0.154 

0.361 

0.121 

0.326 

0.092 

0.151 

0.358 

0.01 

REGION:  East  North  Central 

0.135 

0.342 

0.116 

0.32 

0.057 

0.131 

0.337 

0.012 

REGION:  East  South  Central 

0.092 

0.289 

0.086 

0.28 

0.02 

0.083 

0.276 

0.03 

REGION:  Mid  Atlantic 

0.086 

0.281 

0.109 

0.311 

0.079 

0.089 

0.285 

0.01 

REGION:  Mountain 

0.053 

0.224 

0.066 

0.248 

0.057 

0.055 

0.228 

0.01 

REGION:  New  England 

0.035 

0.185 

0.037 

0.189 

0.01 

0.036 

0.186 

0.003 

REGION:  Other  or  Missing 

0.023 

0.149 

0.027 

0.162 

0.028 

0.023 

0.15 

0.002 

REGION:  Pacific 

0.127 

0.333 

0.133 

0.339 

0.016 

0.125 

0.331 

0.007 

REGION:  South  Atlantic 

0.375 

0.484 

0.344 

0.475 

0.065 

0.386 

0.487 

0.023 

REGION:  West  North  Central 

0.012 

0.107 

0.015 

0.121 

0.031 

0.011 

0.104 

0.006 

REGION:  West  South  Central 

0.061 

0.24 

0.068 

0.252 

0.029 

0.06 

0.238 

0.004 

GENDER:  Female 

0.325 

0.468 

0.357 

0.479 

0.068 

0.32 

0.467 

0.01 

GENDER:  Male 

0.675 

0.468 

0.643 

0.479 

0.068 

0.68 

0.467 

0.01 

POSITION  TYPE:  Administrative 

0.405 

0.491 

0.395 

0.489 

0.022 

0.398 

0.489 

0.016 

POSITION  TYPE:  Other 

0.009 

0.093 

0.013 

0.111 

0.04 

0.009 

0.093 

0.001 

POSITION  TYPE:  Professional 

0.535 

0.499 

0.539 

0.498 

0.008 

0.543 

0.498 

0.015 

POSITION  TYPE:  Technical 

0.051 

0.219 

0.054 

0.225 

0.014 

0.051 

0.22 

0.001 

SUPERVISOR:  All  Other  Positions 

0.932 

0.252 

0.919 

0.273 

0.053 

0.93 

0.256 

0.01 

SUPERVISOR:  Supvisor/Mngr/Ldr 

0.068 

0.252 

0.081 

0.273 

0.053 

0.07 

0.256 

0.01 

ACQ  CAREER  FLD:  Business  (CE  or  FM) 

0.075 

0.264 

0.081 

0.273 

0.023 

0.075 

0.264 

0 
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Unweighted 


Weighted 


Balance  Factors 

37  ACQ  CAREER  FLD:  Contracting 

38  ACQ  CAREER  FLD:  Engineering 

39  ACQ  CAREER  FLD:  Facilities  Engineering 

40  ACQ  CAREER  FLD:  Information  Tech 

41  ACQ  CAREER  FLD:  Life  Cycle  Logistics 

42  ACQ  CAREER  FLD:  Other 

43  ACQ  CAREER  FLD:  Prod,  Quality  &  Manuf 

44  ACQ  CAREER  FLD:  Program  Management 

45  ACQ  CAREER  FLD:  Test  and  Eval 

46  ACQ  CAREER  FLD:  Unknown/Missing 

47  RETIREMENT  PLAN:  CSRS 

48  RETIREMENT  PLAN:  FERS 

49  RETIREMENT  PLAN:  Other 

50  CAREER  LEVEL:  Entry-Level 

51  CAREER  LEVEL:  Mid-Level 

52  CAREER  LEVEL:  Sen/Exec 

53  CAREER  LEVEL:  Uncategorized 

54  YRS  TO  RET  ELIG:  <5  Yrs  to  Go 

55  YRS  TO  RET  ELIG:  10  to  19  Yrs  to  Go 

56  YRS  TO  RET  ELIG:  20+  YRs  to  Go 

57  YRS  TO  RET  ELIG:  5-9  Yrs  to  Go 

58  YRS  TO  RET  ELIG:  Ret  Eligible 

59  PAY  PLAN:  Broad  GS  or  NSPS 

60  PAY  PLAN:  Demo 

61  PAY  PLAN:  Other 

62  HCAP:  Handicapped 

63  HCAP:  Not  Handicapped 


Combination  Pre- 

Policy  Cohort  Group  Policy  Cohort  Group 


Mean 

SD 

Mean 

SD 

0.199 

0.399 

0.16 

0.366 

0.213 

0.409 

0.251 

0.434 

0.063 

0.242 

0.052 

0.221 

0.053 

0.224 

0.055 

0.228 

0.141 

0.348 

0.112 

0.315 

0.026 

0.159 

0.034 

0.182 

0.062 

0.242 

0.059 

0.235 

0.092 

0.29 

0.08 

0.271 

0.039 

0.195 

0.047 

0.211 

0.036 

0.186 

0.069 

0.254 

0.035 

0.184 

0.154 

0.361 

0.955 

0.207 

0.839 

0.368 

0.01 

0.099 

0.007 

0.083 

0.295 

0.456 

0.289 

0.453 

0.597 

0.49 

0.633 

0.482 

0.099 

0.299 

0.069 

0.253 

0.008 

0.09 

0.01 

0.098 

0.068 

0.252 

0.078 

0.268 

0.302 

0.459 

0.324 

0.468 

0.487 

0.5 

0.424 

0.494 

0.111 

0.315 

0.131 

0.338 

0.031 

0.173 

0.042 

0.201 

0.847 

0.36 

0.812 

0.391 

0.139 

0.346 

0.179 

0.383 

0.014 

0.117 

0.009 

0.094 

0.082 

0.275 

0.083 

0.275 

0.918 

0.275 

0.917 

0.275 

Abs  Std 

Diff 

Weighted  Pre-Policy 

Cohort  Group  (Synthetic 

Policy  Cohort  Group  3) 

Abs  Std 

Mean  SD  Diff 

0.099 

0.206 

0.405 

0.018 

0.095 

0.212 

0.408 

0.003 

0.045 

0.06 

0.238 

0.011 

0.008 

0.047 

0.213 

0.026 

0.083 

0.143 

0.35 

0.007 

0.053 

0.026 

0.16 

0.002 

0.015 

0.066 

0.249 

0.016 

0.043 

0.087 

0.282 

0.019 

0.037 

0.04 

0.196 

0.004 

0.178 

0.036 

0.187 

0.003 

0.649 

0.037 

0.19 

0.014 

0.562 

0.954 

0.209 

0.006 

0.029 

0.009 

0.092 

0.014 

0.014 

0.313 

0.464 

0.04 

0.072 

0.586 

0.493 

0.023 

0.102 

0.093 

0.29 

0.021 

0.018 

0.008 

0.087 

0.006 

0.039 

0.065 

0.247 

0.013 

0.048 

0.306 

0.461 

0.008 

0.126 

0.487 

0.5 

0.001 

0.062 

0.114 

0.318 

0.007 

0.066 

0.029 

0.167 

0.012 

0.098 

0.851 

0.357 

0.009 

0.116 

0.14 

0.347 

0.004 

0.042 

0.009 

0.096 

0.04 

0.002 

0.085 

0.279 

0.011 

0.002 

0.915 

0.279 

0.011 
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Figure  G.6.1.  All  Absolute  Standard  Differences  in  Means  are  Below  0.1  After  Weighting 


Figure  G.6.2.  After  Weighting,  Some  Differences  in  Means  are  No  Longer  Statistically  Significant 


esmax.ATT 


Rank  of  p-value  rank  for  pretreatment  variables 
(hollow  is  weighted,  solid  is  unweighted) 
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Appendix  H.  Sensitivity  Analysis:  Lags  Instances  Not  Included  in 
Civilian  DAW  Cohorts 


The  goal  of  this  sensitivity  analysis  is  to  understand  whether  any  of  the  retention  test  results 
are  sensitive  to  the  inclusion  of  the  “lag  instances”  in  the  civilian  DAW  cohorts.104  Each  test 
described  in  Chapters  8  and  9  (and  Appendix  F)  is  run  without  including  the  lag  instances,  and 
this  Appendix  presents  the  results.  If  the  “sensitivity”  test  results  presented  in  this  Appendix  are 
similar  to  the  “original”  test  results  described  in  Chapters  8  and  9  (and  Appendix  F),  then  one 
would  conclude  that  the  tests  are  not  sensitive  to  the  inclusion  of  the  lag  instances. 

Section  H.l,  below,  displays  the  sensitivity  test  results  for  retention  analysis  Part  I.  The 
results  are  displayed  in  Tables  H.l.l-H.1.7.  There  are  two  instances  where  the  original  analysis 
finds  a  statistically  significant  relationship  and  the  sensitivity  analysis  fails  to  find  a  statistically 
significant  relationship  at  a  10  percent  level  of  significance. 

4.  The  original  analysis  on  the  early  pre -policy  cohort  group  finds  that  external  hires  with 
the  Active  Duty  +  Military  DAW  prior  career  profile  are  significantly  more  likely  (at  a  10 
percent  level  of  significance)  to  substantively  transfer  out  of  or  separate  from  the  civilian 
DAW  in  any  given  year  compared  to  external  hires  with  no  prior  DoD  experience  (see 
Table  8.7).  The  sensitivity  test  on  the  early  pre-policy  cohort  group  finds  no  statistically 
significant  difference  in  retention  between  these  two  groups  (see  Table  H.1.7). 

5.  The  original  analysis  on  the  later  pre -policy  cohort  group  finds  that  external  hires  with 
the  Active  Duty  +  Military  DAW  prior  career  profile  are  significantly  less  likely  (at  a  five 
percent  level  of  significance)  to  substantively  transfer  out  of  or  separate  from  the  civilian 
DAW  in  any  given  year  compared  to  external  hires  with  no  prior  DoD  experience  (see 
Table  8.7).  The  sensitivity  test  on  the  later  pre-policy  cohort  group  finds  no  statistically 
significant  difference  in  retention  trends  between  these  two  groups  (see  Table  H.1.7). 

These  two  differences  suggest  that  these  particular  tests  are  slightly  sensitive  to  the  inclusion  of 
the  lag  instances.  However,  the  other  sensitivity  tests  presented  in  Section  H.  1  show  results  that 
are  similar  to  those  from  the  original  analysis.  The  overall  findings  from  retention  analysis  Part  I 
do  not  appear  to  be  sensitive  to  the  inclusion  of  the  lag  instances  in  the  civilian  DAW  cohorts. 

Section  H.2  shows  the  sensitivity  test  results  for  retention  analysis  Part  II.  Tables  H.2.1  and 
H.2.2  show  that  the  results  are  similar  to  those  from  the  original  analysis. 

In  the  end,  except  for  the  two  instances  described  above,  this  sensitivity  analysis  shows  that 
the  retention  tests  presented  in  this  study  are  not  particularly  sensitive  to  the  inclusion  of  the  lag 
instances  in  the  civilian  DAW  cohorts.  The  overall  findings  from  retention  analyses  Parts  I  and  II 
hold  regardless  of  whether  the  lag  instances  are  included. 

104  The  “lag  instances”  are  defined  in  Chapter  6.  Figure  6.2  shows  the  number  of  lag  instances  included  in  each  FY 
cohort. 
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H.1  Sensitivity  Tests  for  Retention  Analysis  Part  I 


Table  H.1.1.  There  is  No  Evidence  Suggesting  that  Entrants  with  prior  DoD  Experience  Have  a 
Lower  Hazard  of  Substantively  Transferring  Out  of  or  Separating  From  the  Civilian  DAW 
Compared  to  Entrants  with  No  Prior  DoD  Experience 


Doubly  Robust  Cox  PH 

Pre-Policy  Cohort  Group 

Early 

Later 

Combination 

N 

25718 

27971 

53689 

Degrees  of  Freedom 

25717 

27970 

53688 

Estimation 

-0.010 

0.033 

0.008 

Hazard  Ratio 

0.990 

1.034 

1.008 

Standard  Error 

0.054 

0.045 

0.037 

p- value 

0.8517 

0.4642 

0.8271 

Table  H.1 .2.  There  is  Some  Evidence  Suggesting  that  Internal  Hires  Have  a  Greater  Hazard  of 
Substantively  Transferring  Out  of  or  Separating  From  the  Civilian  DAW  Compared  to  External 

Hires 


Doubly  Robust  Cox  ph  Pre-Policy  Cohort  Group 


Early 

Later 

Combination 

N 

25718 

27971 

53689 

Degrees  of  Freedom 

25717 

27970 

53688 

Estimation 

-0.015 

0.072 

0.036 

Hazard  Ratio 

0.986 

1.075 

1.037 

Standard  Error 

0.037 

0.028 

0.021 

p-value 

0.6936 

0.0091 

0.0832 
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Table  H.1.3.  Internal  Hires  Have  a  Greater  Hazard  of  Substantively  Transferring  Out  of  the  Civilian 

DAW  Relative  to  External  Hires 


Doubly  Robust  Cox  PH 

Pre-Policy  Cohort  Group 

Early 

Later 

Combination 

N 

25718 

27971 

53689 

Degrees  of  Freedom 

25717 

27970 

53688 

Estimation 

0.243 

0.359 

0.294 

Hazard  Ratio 

1.276 

1.432 

1.342 

Standard  Error 

0.062 

0.047 

0.039 

p-value 

<.0001 

<.0001 

<.0001 

Table  H.1.4.  Internal  Hires  Have  a  Lower  Hazard  of  Separating  From  the  Civilian  DAW  Compared  to 

External  Hires 


Doubly  Robust  Cox  ph  Pre-Policy  Cohort  Group 


Early 

Later 

Combination 

N 

25718 

27971 

53689 

Degrees  of  Freedom 

25717 

27970 

53688 

Estimation 

-0.156 

-0.127 

-0.123 

Hazard  Ratio 

0.856 

0.881 

0.885 

Standard  Error 

0.048 

0.035 

0.027 

p-value 

0.0012 

0.0003 

<.0001 

Table  H.1.5.  Internal  Hires  Have  a  Lower  Hazard  of  Separating  from  the  Wider  DoD  Civilian 

Workforce  Compared  to  External  Hires 


Doubly  Robust  Cox  ph  Pre-Policy  Cohort  Group 


Early 

Later 

Combination 

N 

25718 

27971 

53689 

Degrees  of  Freedom 

25717 

27970 

53688 

Estimation 

-0.128 

-0.142 

-0.126 

Hazard  Ratio 

0.879 

0.867 

0.881 

Standard  Error 

0.042 

0.030 

0.022 

p-value 

0.0023 

<.0001 

<.0001 
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Table  H.1.6.  Among  External  Hires,  Those  with  Prior  DoD  Experience  Have  a  Greater  Hazard  of 
Substantively  Transferring  Out  of  or  Separating  From  the  Civilian  DAW  Compared  to  Those  with 

No  Prior  DoD  Experience 


Doubly  Robust  Cox  ph  Pre-Policy  Cohort  Group 


Early 

Later 

Combination 

N 

10965 

13906 

24871 

Degrees  of  Freedom 

10964 

13905 

24870 

Estimation 

0.109 

0.151 

0.124 

Hazard  Ratio 

1.115 

1.164 

1.132 

Standard  Error 

0.054 

0.054 

0.040 

p-value 

0.0449 

0.0053 

0.0017 

Table  H.1.7.  In  General,  External  Hires  with  a  Prior  Career  Profile  that  Includes  DoD  Experience 
(With  the  Exception  of  the  Active  Duty  +  Military  DAW  Prior  Career  Profile)  Have  a  Greater  Hazard 
of  Substantively  Transferring  Out  of  or  Separating  From  the  Civilian  DAW  Compared  to  External 

Hires  with  No  Prior  DoD  Experience 


Cox  PH  with  Controls 

Hazard  Ratio  Estimation  (ref=‘ 

‘no  prior  DoD  experience”) 

Prior  Career  Profile 

Early 

%  External 

Later 

%  External 

Combination 
%  External 

Hires 

Estimate 

Hires 

Estimate 

Hires 

Estimate 

DoD  Civ 

7.85 

***1.198 

5.79 

“1.152 

6.7 

***1.178 

DoD  Civ  +  Civ  DAW 

6.17 

1.045 

4.16 

“1.166 

5.05 

*1.094 

Active  Duty 

23.01 

***1.154 

23.7 

“‘1.135 

23.4 

***1.14 

Active  Duty  +  DoD  Civ 

1.88 

“1.257 

1.62 

“‘1.367 

1.73 

“‘1.325 

Other  DoD  Experience 

1.81 

***1.461 

2.13 

*“1.394 

1.99 

“*1.415 

Active  Duty  +  Mil  DAW 

5.98 

1.123 

7.87 

0.93 

7.04 

1.008 

p-value<  *0.1,  **0.05,  ‘**0.01 


149 


H.2  Sensitivity  Tests  for  Retention  Analysis  Part  II 


Table  H.2.1.  Individuals  in  the  Synthetic  Pre-Policy  Cohort  Groups  Have  a  Greater  Hazard  of 
Substantively  Transferring  Out  of  or  Separating  From  the  Civilian  DAW  Relative  to  Individuals  in 

the  Policy  Cohort  Group 


Cox  PH 

Synthetic  Early  Pre-Policy 
Cohort  Group3 

Synthetic  Later  Pre-Policy 
Cohort  Groupb 

Synthetic  Combination 
Pre-Policy  Cohort  Group3 

N 

115136 

115136 

115136 

Estimation 

0.123 

0.099 

0.114 

Hazard  Ratio 

1.131 

1.104 

1.120 

Standard  Error 

0.019 

0.014 

0.015 

p- value 

<.0001 

<.0001 

<.0001 

aThe  policy  cohort  group  weighted  to  look  like  the  early  pre-policy  cohort  group 
bThe  policy  cohort  group  weighted  to  look  like  the  later  pre-policy  cohort  group 
cThe  policy  cohort  group  weighted  to  look  like  the  combination  pre-policy  cohort  group 


Table  H.2.2.  Individuals  in  the  Synthetic  Policy  Cohort  Groups  Have  a  Lower  Hazard  of 
Substantively  Transferring  Out  of  or  Separating  From  the  Civilian  DAW  Relative  to  Individuals  in 

the  Pre-Policy  Cohort  Groups 


Cox  PH 

Synthetic  Policy  Cohort 
Group  1a 

Synthetic  Policy  Cohort 
Group  2b 

Synthetic  Policy  Cohort 
Group  3C 

N 

51436 

55942 

107378 

Estimation 

-0.054 

-0.077 

-0.068 

Hazard  Ratio 

0.947 

0.926 

0.934 

Standard  Error 

0.018 

0.015 

0.011 

p- value 

0.0026 

<.0001 

<.0001 

aThe  early  pre-policy  cohort  group  weighted  to  look  like  the  policy  cohort  group 
bThe  later  pre-policy  cohort  group  weighted  to  look  like  the  policy  cohort  group 
cThe  combination  pre-policy  cohort  group  weighted  to  look  like  the  policy  cohort  group 
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Appendix  I.  External  Hires  and  Internal  Hires 


“External  hires”  are  defined  as  civilian  DAW  entrants  who  are  either  in  the  active-duty 
military  (in  a  DAW  position  or  otherwise)  or  not  in  the  DoD  at  all  in  the  FY  prior  to  them  joining 
the  civilian  DAW.  “Internal  hires”  are  defined  as  entrants  who  appear  in  a  wider  non-DAW  DoD 
civilian  position  in  the  FY  prior  to  them  joining  the  civilian  DAW.  These  two  terms  (“external 
hires”  and  “internal  hires”)  are  similar  to  the  two  groups  of  entrants  included  in  a  civilian  DAW 
cohort,  namely  “new  hires”  “substantive  transfers,”  which  are  defined  in  the  Defining  a  Cohort 
section  in  Chapter  6.  However,  the  “external  hire”  and  “internal  hire”  definitions  are  based  on 
the  past-work-experience  characterization  (specifically,  which  workforce  each  entrant  was  in  in 
the  FY  prior  to  him  or  her  joining  the  civilian  DAW),  which  adjusts  for  “double-count”  years 
(FYs  in  which  one  individual  appears  in  both  the  active-duty  data  file  and  the  DoD  civilian  data 
file),  as  outlined  in  Chapter  6.  Therefore,  some  individuals  who  are  included  in  a  cohort  as  a 
transfer  are  actually  counted  as  external  hires.  This,  however,  is  not  common.  Table  1.1,  below, 
shows  the  number  of  civilian  DAW  entrants  in  FY  cohorts  2000-2014,  breaking  each  cohort  up 
(1)  by  new  hires  and  substantive  transfers  and  (2)  by  external  hires  and  internal  hires.  The  table 
shows  that  there  are  only  10  individuals  who  are  included  in  a  civilian  DAW  cohort  as  a  transfer 
but  are  counted  as  an  external  hire.  In  all  other  cases,  those  who  are  included  in  a  cohort  as  a 
transfer  are  counted  as  an  internal  hire  and  those  who  are  included  in  a  cohort  as  a  new  hire  are 
counted  as  an  external  hire. 

Table  1.1.  For  Several  Cohorts,  a  Few  Individuals  Who  are  Included  in  the  Cohort  as  a  Transfer  are 

Counted  as  an  External  Hire 


FY  Cohort 

New  Hire 
(A) 

Transfer 

(B) 

External  Hire 
(C) 

Internal  Hire 
(D) 

TOTAL 

ENTRANTS 

C-A 

D-B 

2000 

2271 

2162 

2271 

2162 

4433 

0 

0 

2001 

3238 

5249 

3238 

5249 

8487 

0 

0 

2002 

4779 

5577 

4779 

5577 

10356 

0 

0 

2003 

4912 

4054 

4912 

4054 

8966 

0 

0 

2004 

4975 

5742 

4978 

5739 

10717 

3 

-3 

2005 

5052 

5168 

5052 

5168 

10220 

0 

0 

2006 

4691 

3353 

4692 

3352 

8044 

1 

-1 

2007 

5309 

3371 

5310 

3370 

8680 

1 

-1 

2008 

8231 

4217 

8233 

4215 

12448 

2 

-2 

2009 

12975 

4333 

12976 

4332 

17308 

1 

-1 
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I - 1 - 1 - 1 - 1 - 1 


FY  Cohort 

New  Hire 
(A) 

Transfer 

(B) 

External  Hire 
(C) 

Internal  Hire 
(D) 

TOTAL 

ENTRANTS 

C-A 

D-B 

2010 

14814 

4259 

14816 

4257 

19073 

2 

-2 

2011 

8675 

4168 

8675 

4168 

12843 

0 

0 

2012 

6053 

3685 

6053 

3685 

9738 

0 

0 

2013 

4750 

3269 

4750 

3269 

8019 

0 

0 

2014 

5052 

3204 

5052 

3204 

8256 

0 

0 
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Appendix  J.  Career  Level  Definitions 


RAND  researchers  created  the  career-level  descriptors  as  a  way  to  have  approximate  uniform 
workforce  levels  across  different  pay  plans  (Guo,  Hall-Partyka  and  Gates,  2014).  The  career 
levels  and  the  pay  plans,  grades,  and  steps  that  make  up  each  career  level  are  listed  in  a  2012 
RAND  Project  Memorandum  that  is  not  cleared  for  open  publication  (Gates  et  ah,  2012,  pp.  31- 
32).  Below  is  an  excerpt  from  that  Project  Memorandum  that  defines  the  career  levels: 

Entry  level  includes  GS  (and  related)  1-8;  WG,  WD,  WY,  XF,  WK  1-8;  YA, 

YB,  YP,  YD,  YE,YH,  YI,  YK,  YL,  YM,  DR  1;  NM  2;  DA,  DB,  DE,  DJ,  DP, 

ND,  NH,  NJ,  NK,  NO,  NP,  NR,  NT  1-2;  DK  1-3.  For  the  TP  pay-plan  workers 
are  coded  by  grade  and  step.  Entry-level  includes  all  AA  workers,  and  steps  1-5 
for  the  C*,  D*,  E*,  and  F*  grades.  In  addition,  all  WT  workers  and  workers  in 
the  NFI  and  DR  pay-plan  are  at  grade  0. 

Mid  level  includes:  GS  (and  related)  9-13;  WG,  WD,  WY,  XF,  WK  9-15; 

WL,  XG,  WR  1-7,  WS,  WJ,  WQ,  XH,  WA  1-7;  YC,  YF,  YJ,  YN  -1;  YA,  YB, 

YD,  YE,  YH,  YI,  YK,  YL,  YM,  DR  -2;  YB,  YE,  YL,  DB,  DE,  DJ,  DP,  NH,  NK, 

NO,  NP,  NM-3;  DK-4;  DA,  ND,  NJ,  NR,  NT  3-4;  1A-2-3.  For  the  TP  pay-plan 
workers  are  coded  by  grade  and  step.  Mid-level  includes  steps  6-10  for  the  C*, 

D*,  E*,  and  F*  grades. 

Senior  level  includes  GS  (and  related)  14-15;  WL,  XG,  WR,  WS,  WJ,  WQ, 

XH,  WA  8  and  above;  YC,  YF,  YJ,  YN  2  and  3;  YA,  YD,  YH,  YI,  YK,  YM  3; 

YB,  YE,  YL,  DB,  NH,  NP  4;  DE,  DJ,  1A,  NO,  NM  4-5,  DR  3-4;  DP,  NR,  ND  5; 

DA,  NT  5-6;  IP  (all).  We  have  also  tentative  coded  all  members  of  the  YJ 
(Medical  supervisors)  and  YG  (Medical  physicians/dentists)  as  senior  pending 
further  input.  For  the  TP  pay-plan  workers  are  coded  by  grade  and  step.  Senior- 
level  includes  steps  1 1  and  above  for  the  C*,  D*,  E*,  and  F*  grades,  as  well  as 
all  steps  for  the  L*,  O*,  and  P*  grades. 

Executive  includes:  ES  (all);  DB  5-6;  ND  6;  NP  5;  IE  (all);  For  the  TP  pay- 
plan  we  include  those  in  the-K*  grade. 

These  categories  may  be  improved  based  on  feedback  from  FCMs. 

Uncategorized  employees  either  have  a  grade-level  that  is  not  included  in  our 
career-level  definitions  or  are  in  a  pay  plan  for  which  we  do  not  have  career-level 
information  (AD,  CA,  DG,  DS,  DT,  ED,  EE,  EF,  EG,  EH,  El,  EX,  NC,  NG,  SL, 

SR,  ST,  WB,  WM,  WN,  WO,  WU,  XP,  ZZ)  (Gates  et  al„  2012,  pp.  31-32). 

For  this  study,  an  entrant’s  “Career  Level  Upon  Entrance,”  as  listed  in  Table  6.2,  is  the  career 
level  an  entrant  is  recorded  in  in  his  or  her  civilian  DAW  cohort  year  (for  that  particular  instance, 
as  some  individuals  are  in  multiple  civilian  DAW  cohorts). 
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